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SUMMARY 

PROBLEM 

The  current  training1  of  CIC  (Combat  Information  Center)  teams  is  ooatly 
and  often  inefficient  and  unsatisfying.  A  large  number  of  interacting  team 
members  must  be  present,  oostly  equipment  is  involved,  and  a  high  ratio  of 
instructors  to  trainees  is  required.  Existing  training  system  designs  are 
inadequate  to  handle  a  training  situation  where  a  team  must  interaot  mainly 
through  voioe  communication  while  pursuing  its  goal  in  an  unstructured  and 
unpredictable  environment.  The  area  is  ripe  for  the  application  of  emerging 
advanced  technologies. 

PURPOSE 

The  purpose  of  this  study  is  to  develop  a  preliminary  system  oonoept 
for  a  stand-alone  team  training  demonstration  system  whioh  will  support  reaearoh 
into  team  training  issues.  A  significant  part  of  this  development  has  involved 
extensive  front-end  analysis  to  oomprehend  the  scope  of  the  team  training 
problem.  Major  design  emphases  have  included  the  oombined  application  of 
adaptive  training,  automated  performance  measurement,  and  automated  speeoh 
recognition  and  speech  synthesis. 

approach 

To  get  to  the  root  of  team  training  issues,  extensive  analyses  of  several 
relevant  facets  of  team  training  were  oarried  out.  These  investigations 
fooused  on  our  rent  team  training,  the  operational  environment  in  which  teams 
perform,  team  training  research  issues,  potential  training  approaches,  team 
communications,  and  performance  measurement  for  teams.  A  three  member  Anti- 
Submarine  Warfare  (ASW)  team  aboard  a  shipboard  CIC  provided  a  oonorete  basis 
for  the  analyses;  however .  it  appears  that  this  selection  results  in  little 
loss  of  generality  and  that  the  results  are  applicable  to  many  different 
kinds  of  military  teams.  In  particular,  the  analyses  are  relevant  to  other 
CIC  teams. 

The  system  oonoept  which  emerged  from  these  analyses  oalls  for  ooraputer 
simulation  of  several  members  of  the  full  ASW  team  with  automated  speeoh 
recognition  and  speech  generation  providing  oommunioation  with  the  simulated 
supporting  personnel.  Most  significantly,  the  system  oonoept  provides  for 
the  team  training  system  to  funotion  with  one  or  two  of  the  human  learn  members 
absent;  the  role  of  the  missing  team  members  would  be  played  by  sophisticated 
know ledge- based  computer  models. 

FINDINGS 

This  report  presents  the  results  of  an  extensive  analysis  of  the  team 
training  problem  in  an  ASW  context.  Based  on  this  analysis,  a  system  oonoept 
was  developed  to  support  the  design  of  a  team  training  demonstration  system. 
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A  functional  description  of  Mils  system  vas  prepared,  acd  a  staged  imp!  mutation 
plan  was  put  together.  It  is  believed  that  toe  syatw  u  described  is  feasible, 
fil'd  that  the”*  is  a  strong  need  for  the  kind  of  tool  it  will  provide.  It 
is  recommended  the  system  be  developed  according  to  the  staged  Implement¬ 
ation  plan  which  is  presented.  Some  shor  t,  term  Motions  arc  roooamended. 
These  lAoiLude;  (1)  careful  monitoring  and  oontiruieg  development  of  automated 
speech  technologies,  (2)  earl;  development  of  a  knccledge-based  system  prototype, 
and  (3)  enhancement  of  existing  training  by  re-emphasising  communications 
in  the  oiirrent  training  syllabi. 
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SECTION  I 
INTRODUCTION 


PROBLEM 

Tactical  decision  and  control  in  Navy  Combat  Information  Canters  require 
the  operators  to  communicate  with  each  other  and  with  external  personnel. 
While  some  oommuni cation  takes  place  through  the  Naval  Tactical  Data  System 
(NTDS),  most  team  r  _>nmunioations  are  verbal.  Sinoe  these  communications 
represent  most  of  the  team  interaction,  it  is  understandable  that  team  perfor¬ 
mance,  and  therefore  mission  performance,  der^nds  on  the  quality  of  'eae 
communications.  An  analysis  of  CIC  communications  (see  Appendix  ’  ~ts 
that  informative  and  timely  communications  are  essential  to  good  team  ^eua  .or 
and  mission  success,  while  lack  of  good  communication  is  likely  to  lead  to 
mission  failure. 

The  training  of  CIC  teams  requires  a  large  number  of  interacting  team 
members  to  be  present,  involves  costly  equipment,  and  demands  a  hi^h  ratio 
of  instructors  to  trainees.  It  is  a  costly  and,  unfortunately,  often  inefficient 
operation,  since,  for  example,  the  least  skilled  team  members  can  prolong 
training  for  everyone.  Team  training,  in  3pite  of  the  name,  does  not  usually 
involve  formal  training  in  team  skills.  Because  of  insufficient  or  inappropriate 
communications,  the  trainees  often  simply  do  not  perform  as  a  team  for  much 
of  initial  training. 

Clearly,  team  training  is  worthy  of  the  application  of  advanced  technologies 
and  of  researoh  into  improved  team  training  techniques.  However,  existing 
training  systems,  or  training  research  systems,  do  not  provide  for  automation 
with  multiple  trainees  interacting  mainly  through  speech.  One  cannot  hope 
to  develop  automated  training  techniques  applicable  to  CIC  team  training 
without  having  an  automated  speech  understanding  capability.  Nor  can  one 
advance  team  training  methods  through  research  in  critical  areas  such  as 
team  characteristics,  communications,  feedback,  training  sequence,  task  environ¬ 
ment,  and  performance  measurement  with  more  traditional  techniques.  Application 
and  extension  of  the  emerging  advanced  technologies  is  very  much  in  order. 

PURPOSE 

The  purpose  of  the  current  study  is  to  perform  front-end  analyses  for 
preliminary  design  and  determination  of  feasibility  of  a  stand-alone  research 
training  device.  Major  design  emphases  are  the  combined  application  of  adaptive 
training,  automated  performance  measurement,  and  computer  voice  synthesis 
and  recognition. 
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APPROACH 

The  front-end  analyses  Included: 

•  current  team  training 

•  operational  environment 

e  team  training  research  issues 

•  training  approach 

e  team  communications 

•  models  of  operator  behavior 

•  performance  measurement 

e  hardware  and  software  requirements 

While  it  is  intended  to  extend  the  applioation  of  emerging  technologies 
to  a  variety  of  military  teams,  these  front-end  analyses  require  the  analyst 
to  get  very  specific  if  the  end  result  is  to  be  useful.  For  this  reason, 
a  specifio  applioation—tbe  Anti-Submarine  Warfare  (ASW)  team  in  a  shipboard 
CIC— was  selected.  However,  it  is  believed  that  this  selection  results  in 
little  loss  of  generality,  especially  to  other  teams  in  the  CIC  (see  Appendix 
K). 

More  specifically,  the  ASW  team  of  a  DD-963  (Spruanoe  olaos  destroyer) 
is  assumed  for  analytic  purposes.  It  is  assumed  that  at  most  three  members 
are  physically  present  for  training;  these  are  the  ASW  Operations  Coordinator 
(ASWOC),  the  Anti-Submarine  Air  Controller  (ASAC),  and  the  ASW  Fire  Control 
Offioer  ( ASWFCO) *  All  the  other  ASW  team  members  are  simulated  using  oomputer 
models  of  their  behavior  whioh  employ  automated  speech  recognition  and  speech 
generation.  Furthermore,  eaoh  of  the  three  team  members  may  also  be  modeled, 
so  that  at  the  extreme,  team  training  may  be  attempted  with  only  one  trainee. 

The  design  orientation  calls  for  ail  models  to  perform  at  a  high  level 
of  competency,  and,  therefore,  the  model-ASWOC  (for  example)  may  be  compared 
to  the  trainee- ASWOC.  Provision  for  lesser  levels  of  model  performance  is 
required  to  permit  future  training  with  deteriorated  teen  performance.  Of 
oourse,  simulator  functions  (auoh  as  vehicle  motion,  support  personnel ,  and 
performance  measurement)  must  be  activated  by  apeeoh  recognition  when  oti.er 
user  inputs  oannot  be  used.  (A  more  detailed  description  of  the  system  oonoept 
is  presented  in  Section  IV.) 
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BACKGROUND 

In  the  past,  the  emerging  technologies  have  been  employed  in  the  attempt 
to  improve  flight  training  and  ground  controller  training  systems,  but  it 
is  expected  that  the  approach  can  be  extended  to  military  tesc.  training  systems. 
The  development  of  these  subsystems  is  charted  by  a  series  of  Navy  training 
programs  (see  Appendix  A): 

e  Tactical  Advanced  Combat  Directions  and  Electronic  Warfare  (TACDEW) 
(involves  simulation,  replay,  very  simple  performance  measurement); 

e  Adaptive  Flight  Training  System  (AFTS)  (involves  simulation,  replay, 
performance  measurement,  simple  feedback,  simple  adaptive  logic); 

e  Ground  Controlled  Approach  -  Controller  '’’raining  System  (GCA-CTS) 
(involves  simulation,  replay,  performance  measurement,  feedback,  adaptive 
logic,  simple  computer  aided  instruction,  simple  diagnosis  and  remedi- 
ation) ; 

e  Air  Controller  Exerciser  (ACE)  (involves  extensions  of  all  the  foregoing). 

In  terms  of  previous  system  designs,  then,  what  is  different  about  the 
current  program?  One  difference  is  that  the  "trainee"  is  a  "team,"  and  that 
communications  is  a  fooal  issue.  A  particularly  important  and  basic  difference, 
which  will  significantly  affeot  the  design,  la  that  the  military  team  situation 
is  rather  unspecific  and  unpredictable  oompared  to  previous  applications. 
We  are  therefore  challenged  to  push  the  state  of  the  training  technology 
to  provide  solutions  which  heretofore  have  not  been  available:  the  design 
of  automated  training  as  applied  to  team  performance,  foousing  on  communications 
control,  and  in  a  dynamic  (CIC)  environment. 

FOCUS  OF  THIS  REPORT 

The  main  body  of  this  report  disousses  general  aspects  of  the  team  training 
system.  This  discussion  begins  with  the  analysis  of  team  training  presented 
in  Section  II  and  continues  with  a  disoussion  of  the  theoretical  researoh 
issues  in  Section  III.  Section  IV  describes  the  system  oonoept  at  a  general 
level,  and  Section  V  discusses  some  of  the  technological  research  issues 
involved  in  the  implementation.  Section  VI  provides  system  functional  require¬ 
ments,  while  Section  VII  provides  a  suggested  implementation  plan.  The  report 
concludes  with  conclusions  and  recommendations  in  Section  VIII. 

The  body  of  the  report  is  immediately  followed,  as  required,  by  the 
list  of  references.  This  organization  is  a  bit  awkward,  since  most  of  the 
references  to  the  literature  occur  in  the  more  teohnioal  appendices.  It 
was  considered  advisable,  nonetheless,  to  consolidate  all  references  here, 
rather  than  provide  separate  reference  liats  for  the  appendices. 
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The  bulk  of  technical  detail  has  been  relegated  to  the  appendioes,  two 
of  whloh  have  been  published  separately  because  of  their  confidential  classi¬ 
fication.  Appendix  A  discusses  training  technology.  Appendix  B  provides 
a  comprehensive  review  of  the  literature  on  team  training.  Appendix  C  (separately 
bound)  provides  a  scenario  for  the  envisioned  team  training  system.  Appendix 
D  provides  a  graphic  representation  of  the  high  level  knowledge  of  one  team 
member  (the  ASVOC).  Appendix  E  describes  the  functional  characteristics 
of  the  training  consoles.  Appendix  F  lists  the  ootaaunioations  circuits  used 
by  the  team  members.  Appendix  G  provides  the  automated  speech  technology 
requirements.  Appendix  H  describes  software  design  considerations,  while 
Appendix  I  offers  a  more  detailed  design  for  the  knowledge-based  models  recom¬ 
mended  for  the  system.  Appendix  J  provides  a  most  interesting  analysis  of 
the  communications  used  in  three  tape-recorded  training  sessions,  appendix 
X  offers  a  description  of  the  generic  aspeots  of  Anti-Air  Warfare  (AAW)  training 
whloh  suggest  that  the  concepts  developed  i^re  may  he  applicable  to  a  broad 
range  of  military  training  situations.  Appendix  L,  also  separately  bound, 
provides  the  evaluation  sheets  presently  used  in  evaluating  team  training 
to  show  the  emphasis  plaoed  upon  good  communications.  A  Glossary  of  terns 
and  aoronyms  concludes  ths  report. 
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SECTION  II 
TRAINING  ANALYSIS 

A  training  analysis  was  conducted  for  ASW  Team  Training  and  is  presented 
in  three  parts:  (1)  observations  of  fleet  team  training,  (2)  task  analysis, 
and  (3)  performance  measurement  analysis. 

OBSERVATIONS  OP  TEAM  TRAINING 

It  had  been  initially  intended  to  collect  training  and  task  data  for 
DD-963  and  FFG-7  class  ships.  However,  training  personnel  convinced  us  that 
a  compatible  analysis  was  not  possible  between  the  two  ships,  and  furthermore, 
the  DD-963  presented  a  better  ASW  subject. 

Consequently,  observations  of  team  training  were  limited  to  the  DD-963* 
ASW  training  was  observed  in  two  DD-963  ships  dockside,  multi-threat  exercises 
at  the  Fleet  Combat  Training  Center,  Atlantic,  and  training  sessions  at  the 
Fleet  Combat  Training  Center,  Pacific.  One  of  the  dockside  sessions  was 
oonduoted  primarily  for  diagnostic  purposes  to  prescribe  needed  classroom 
training,  and  the  multi- threat  exercise  did  not  inolude  ASW  (although  the 
team  training  was  highly  analogous  and  informative) .  Therefore,  the  following 
discussion  will  emphasize  the  specific  information  collected  from  one  three- 
day  dockside  training  exercise  aid  one  three-day  simulator  training  exercise. 
To  avoid  redundant  descriptions,  the  presentation  will  center  on  the  dookside 
training,  and  occasional  references  will  be  made  to  the  simulator  training 
where  additional  details  should  be  noted. 

TRAINING  PROCEDURE*  Dockside  training  was  oonduoted  in  the  ship’s  Combat 
Information  Center,  using  the  ship’s  oomputer  and  all  normal  work  stations. 
Speoial  training  software  was  loaded  into  the  computer  to  permit  the  oonduot 
of  training  exeroises  as  specified  by  the  instructional  team.  Eight  instructors 
from  the  Fleet  Anti-Submarine  Warfare  Training  Center,  Paoific,  oonduoted 
the  training.  All  except  the  training  supervisor  manned  stations  to  perform 
the  functions  of  helioopter  pilots,  Soene  of  Action  Commander,  sister  ship, 
submarine  maneuvering,  and  the  like.  Ship  maneuvering  commands  were  transmitted 
to  the  bridge  where  course  changes  were  entered  manually  to  cause  heading 
and  speed  instruments  to  indicate  properly. 

The  Operations  Summary  Console  (OSC)  was  not  functioning  throughout 
the  training  exercises,  so  these  functions  were  performed  manually  on  the 
Dead  Reckoning  Tracer.  The  team  under  training  included  the  Anti-Submarine 
Warfare  Operations  Coordinator,  North  and  South  Plotters,  R'T  (Radio/Telephone) 
Talker,  Anti-Submarine  Air  Controller,  Anti-Submarine  Warfare  Fire  Control 
Officer,  Surface  Tracker,  and  Sonar  Operators  (in  an  adjacent  room). 

As  the  computer  was  not  functioning  properly  at  first,  a  lecture  was 
delivered  at  the  outset;  however,  this  lecture  would  have  been  delivered 
at  some  point.  The  leoture  included  a  "walk- through"  of  an  ASW  exeroise, 


NAVTRAEQUIPCEN  80-00095-1 


highlighting  critioal  definitions  and  procedures,  reviewing  appropriate  taotica, 
providing  speoifio  recommendations  for  taotios  during  the  exercises  to  come, 
and  identifying  specif io  dos  and  don'ts.  No  speoifio  instruction  was  given 
with  regard  to  team  interaction,  except  to  generally  encourage  team  members 
to  talk  to  each  other  for  better  understanding  of  their  lnterdependenoy. 

Information  was  given  over  the  loudspeaker  with  regard  to  the  problem 
setup  prior  to  starting  the  simulation.  This  information  inoluded  ship's 
heading  and  speed,  the  mainbody  unit  being  esoorted  intelligence  about  possible 
submarine  enoounters,  sonar  conditions,  and  helicopters  and  weapons  under 
oontrol. 

Each  exeroise  was  oonduoted  largely  without  instructor  intervention, 
although  the  instructors  oould  modify  the  training  problem  and  direot  the 
problem  if  the  trainees  were  having  trouble.  Note  ttat  subtle  instructor 
interaction  was  possible:  for  example,  the  ASAC  was  talcing  to  a  simulated 
helicopter  pilot  who  was,  in  faot,  an  instruetor-ASAC. 

A  debriefing  session  followed  each  exercise.  The  primary  debriefing 
was  held  by  the  training  supervisor  av  the  Dead  Reckoning  Traoer  (DRT) . 
His  oomments  were  directed  primarily  to  the  ASVOC,  the  plotters  and  the  R/T 
talker,  and  others  were  called  to  the  table  as  required.  The  ASUFCO  and 
ASAC  may  have  had  some  interaction  with  their  instructor  counterparts  after 
an  exercise,  but  this  was  not  apparent. 

Exeroises  were  planned  for  a  three-day  period.  The  exercises  proceeded 
through  the  following  sequence  (interestingly,  adding  assets  inoreased  problem 
difficulty) : 

1 .  ownship 

2.  ownship  +  LAMPS  helioopter 

3.  dual  ship 

4.  ownship  +  LAMPS  helioopter  +  dipping  helioopter 

5.  dual  ship  +  LAMPS  helicopter  +  dipping  helicopter 

These  exercises  were  often  repeated,  but  ordinarily  this  was  because  of  team 
member  substitutions.  There  were  two  ASWOCs,  two  ASACs,  and  several  R/T 
Talkers  being  trained.  All  of  the  team  members  had  other  duties  to  perform 
on  ship,  and  the  composition  of  crews  ohanged  with  eaoh  exeroise. 

Task  diffi  i.ty  was  ohanged  in  other  ways:  difficulty  ohanged  with 
types  and  qua  .ties  of  weapons  available,  position  and  type  of  submarine, 
and  restrictions  upon  escorting  and  maneuvering  of  the  mainbody  units.  For 
example,  the  DD-963  might  be  required  to  maintain  station  near  the  mainbody, 
and  might  not  be  allowed  to  significantly  maneuver  the  mainbody  out  of  trouble. 
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Simulator  training  was  administered  in  the  following  similar  sequence; 

1 .  Target  Maneuvering  Analysis  (TMA)  Basics 

2.  Sonar  Mockup  Familiarization  (Sonar  Technicians  (ST)  only) 

3.  Surfaoe  Attack  Unit  (SAU)  Procedures 
i|.  LAMPS 

5.  Mockup  Familiarization 

6.  Passive  Target  Maneuvering  Analysis 

7.  Single  Ship  ASW 

8.  SAU/Mainbody  Operations 

9.  SAU/LAMPS  Operations 

10.  SAU/Mainbody/Multi-Airoraft  Operations 

It  is  understood,  however,  that  the  two  sessions  dealing  with  Target 
Maneuvering  Analysis  have  been  subsequently  deleted  from  the  simulator  schedule. 
Passive  techniques  were  emphasized  to  a  greater  extent  with  the  simulator 
than  with  the  dockside  training. 

RESULTS.  At  the  beginning,  even  with  limited  assets  to  control  and  with 
a  submarine  traveling  in  a  straight  line,  the  situation  was  ohaotic.  However, 
by  the  end  of  the  three-day  session,  the  ASW  team  performance  was  reasonably 
good,  even  with  a  complex  setup  (note,  however,  maneuvering  of  the  adversary 
submarine  was  always  limited).  In  short,  learning  seemed  to  take  plaoe. 

The  items  in  Table  1  are  errors  gleaned  from  the  debriefings  following 
each  exercise;  they  show  the  types  of  difficulties  as  emphasized  by  the  instruc¬ 
tors.  Note  that  the  emphasis  is  on  tactioal  errors,  safety,  time  efficiency, 
and  routine  procedures;  there  was  virtually  no  feedbaok  on  team  behavior. 
There  was  some  reinforcement  of  desirable  performance,  but  feedback  was  primarily 
negative. 

Informal  debriefing  in  the  observed  simulator  exercises  was  quite  similar 
to  that  in  the  dookside  training,  Little  direct  emphasis  was  placed  on  team 
communications,  except  that  in  one  case  debriefing  did  make  a  strong  issue 
of  ASWOC-ASAC  communications  which  had  been  almost  totally  absent.  Also, 
it  is  clear  from  the  formal  evaluation  sheets  used  in  the  simulator  training 
(reproduced  in  Appendix  L),  that  team  communications  are  strongly  emphasized 
and  specifically  evaluated. 
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TABLE  1.  DEBRIEFING  COMMENTS. 

ASAC— didn't  announce  that  the  helioopter  was  out  of  the  weapon  danger  area 
(held  up  by  prosecution  of  attack). 

ASWOC— during  an  urgent  attack,  used  a  torpedo  instead  of  a  weapon  that  could 
get  there  quicker. 

ASWOC— didn't  upgrade  the  classification  of  the  submarine,  affecting  the 
possibility  of  weapons  free  for  attaok. 

«JW0C— didn't  make  correct  turn  and  endangered  ship. 

ASAC— should  have  directed  helioopter  down  track  of  submarine. 

ASAC—oould  have  prosecuted  urgent  attack  quioker  with  the  helioopter. 

ASAC— ordered  helioopter  to  drop  weapon,  but  did  it  without  proper  preliminary 
communication,  so  that  the  helioopter  balked. 

ASWOC— turned  parallel  to  submarine  traok  (good). 

Plotter— inoorreotly  oomputed  Estimated  Time  of  Arrival  (ETA). 

Plotter— confused  magnetic  bearing  with  true  bearing. 

Talker-confusion  as  to  when  to  use  yards  or  miles. 

ASWOC— made  ineffective  use  of  assist  ship. 

ASWOC— didn't  turn  mainbody  out  of  danger. 

ASWOC— brought  helioopter  baok  prematurely  with  weapon  unexpended. 

Plotter— didn't  plot  dipping  helicopter. 

ASWOC— got  ownship  out  of  position. 

Plotter— delayed  in  plotting  possible  submarine  oouraes. 

Talker— said  "port"  instead  of  "starboard,"  used  wrong  oode  name  for  weapon, 
required  exoessive  direction  (new  man). 

ASWOC/Plotter—  new  range  of  predicted  courses  not  plotted  when  the  submarine 
turned  toward  the  mainbody. 
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Informal  discussion  with  training  personnel  indicated  aome  unhappineaa 
with  the  ahipboard  training  procedure.  Equipment  8eldom  worked  properly 
in  the  training  mode,  and  much  equipment  waa  down  routinely.  Furthermore, 
on  the  ahip  the  duties  of  the  crew  did  not  ceaae  for  training,  preventing 
continuity  of  training  with  an  intact  team.  Also,  the  visiting  inatructors 
felt  somewhat  awkward  in  their  training  role,  for  there  were  highly  qualified 
personnel  onboard  who  were,  in  their  opinion,  capable  of  giving  the  training 
(e.g.,  the  CIC  supervisor  was  an  experienced  ohief).  Finally,  although  time 
is  normally  allocated  for  ship  training  on  a  daily  basis,  it  often  had  to 
be  used  for  the  accomplishment  of  other  ship  duties. 

TASK  ANALYSIS 

MISSION  DESCRIPTION.  A  description  of  the  operational  environment  is  given 
in  Appendix  C,  but  the  following  description  is  given  to  provide  a  background 
for  the  ASW  team  tasks.  This  description  is  intended  to  provide  a  setting 
for  the  subsequent  analysis  of  these  team  tasks. 

The  ASW  ships  will  normally  be  esoorting  a  mainbody  of  other  ships, 
probably  including  an  aircraft  carrier.  The  close  escort  funotion  will  continue 
until  the  presenoe  of  an  underwater  threat  is  discovered  or  suspeoted.  At 
this  time  a  datum  is  established— a  position  at  a  given  time  given  along 
with  a  probable  error.  However,  the  submarine  oan  continue  to  move  at  rates 
up  to  its  maximum  speed  in  any  direction  from  the  datum.  Therefore,  a  circle 
is  plotted  centered  on  the  datum  which  represents  an  t.rea  containing  the 
submarine.  This  circle  must  be  replotted  periodically  with  a  larger  radius. 

Thu  ASW  team  can  also  make  the  assumption  that  the  threat,  if  real, 
will  attempt  to  make  an  intercept  with  the  mainbody.  The  possible  approaches 
to  the  mainbody  can  be  estimated  given  some  assumptions  about  the  minimum 
and  maximum  speed  of  the  submarine  threat,  uiven  this  type  of  analysis, 
the  team  oan  make  some  informed  statements  about  the  probable  looation  of 
the  submarine  over  time  (i.e.,  within  an  expanding  circle  and  possible  intercept 
paths) ,  and  make  some  intelligent  decisions  with  regard  to  methods  of  searching 
for  and  intercepting  the  threat. 

A  Search  and  Attack  Unit  will  be  formed  to  engage  the  threat  under  the 
command  of  the  Scene  of  Action  Commander  (SAC),  probably  the  SAU  Commander. 
The  ships,  thus  detached,  will  proceed  in  formation  to  intercept  the  threat; 
the  formation,  line  of  approach,  and  speed  are  decisions  to  be  made  considering 
the  nearness  of  the  threat,  the  available  assets,  and  the  characteristics 
of  the  ship's  sonar  equipment. 

A  characteristic  of  ASW  missions  is  that  sonar  or  visual  oontact  with 
the  chreat  is  infrequent  and  intermittent.  The  ASW  team  must  always  have 
a  plan  ready  for  search  and  another  for  attack:  these  must  be  determined 
in  advance  and  must  be  communicated  to  the  entire  ASW  force.  These  plans 
are  selected  from  published  tactios,  and  the  selection  is  ordinarily  made 
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from  a  small  set  of  likely  candidates.  The  plans  will  make  clear  to  the 
SAD  the  conditions  under  which  eaob  ship  should  take  aotion  and  are  designed 
to  avoid  mutual  interference. 

If  air  assets,  such  as  helicopters,  are  detached  with  the  SAU,  these 
units  can  aot  as  long-distanoe,  high-speed  arms  for  seoroh  and  attack.  Again- 
published  tactics  are  available  for  selection  of  both  search  and  attack. 
Often  the  use  of  the  helioopter  units  for  search  will  involve  laying  down 
a  complex  pattern  of  sonar  units.  Also,  helioopter  detection  may  oome  before 
the  ships  can  arrive  on  the  soene  and,  therefore,  may  also  be  used  to  deliver 
a  weapon. 

The  ships  may  oontinue  to  approach  the  threat,  unless  helicopter  aotion 
ooours  first,  or  unless  the  SAU  is  required  to  stay  close  to  the  malnbody. 
If  t'e  ships  arrive  within  the  range  of  enemy  weapons,  they  must  maneuver 
evasively  to  prevent  the  submarine  from  obtaining  a  fire  oontrol  solution. 
Under  some  oiroumstanoes,  the  ship  may  attaok  immediately,  even  though  the 
probability  of  kill  is  low;  otherwise,  the  ship  will  select  a  weapon  from 
the  available  arsenal  and  wait  for  a  carefully  calculated  fire  oontrol  solution. 

During  attaok,  the  ASV  team  is  responsible  for  keeping  the  assets  of 
the  SAU  dear  of  weapon  danger  areas.  For  example,  the  helioopter  units 
must  be  kept  clear  of  air  launohed  weapons,  and  preferably  should  be  dispatohed 
to  a  good  backup  position. 

Note  that  the  initiation  of  an  attaok  is  oarefully  constrained  and  controlled 
by  higher-command  authority.  There  are  specifio  conditions  established  under 
whioh  a  "weapons  free"  condition  is  established,  and  the  ASV  team  has  an 
obligation  to  communioate  information  which  will  aff'.ot  the  classification 
of  the  threat.  Under  some  conditions,  no  attaok  will  take  plaoe,  and  the 
mission  will  be  solely  to  traok  or  "hold"  the  oontaot  without  closing  on 
it. 

STATE  DIAGRAM.  A  diagram  of  the  above  conditions  is  preser'  ..d  in  Figure 
1.  Each  major  activity  is  presented  as  a  "state"  with  key  deoiuiuns  indicating 
when  the  transition  from  one  state  to  another  will  ooour. 

A  state  diagram  can  be  drawn  for  each  asset  in  the  SAU;  a  diagram  is 
presented  which  can  be  representative  of  either  sister  ship  and  another  whioh 
oan  be  representative  of  each  air  asset. 


These  state  diagrams  serve  to  indioate  the  position  of  ASV  on  the  estab¬ 
lished/emergent  continuum,  as  discussed  elsewhere  (see  Appendix  B).  While 
there  are  often  guidelines  which  strongly  define  oourses  of  aotion,  there 
are  not  rigid  procedures  within  any  state. 

For  example,  a  speoiflc  approach  oourae  may  be  selected  for  a  number 
of  reasons  so  as  to  make  it  appear  that  the  threat  has  not  been  detected; 
an  attack  may  be  prematurely  launched  to  put  the  submarine  on  the  defensive; 
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or  the  decision  to  attack  with  air  assets  or  ship  assets  may  depend  upon 
the  policy  of  the  specific  ship's  commanding  officer.  In  the  process  of 
localizing  the  threat,  oontact  can  be  established,  then  contact  can  be  later 
lost  during  attack,  then  regained  and  attack  again  commenced,  etc.  The  other 
ship  might  get  "hot"  and  attaok  ship/assist  ship  roles  will  change;  the  ASWOC 
may  choose  to  do  something  unusual,  like  putting  the  submarine  in  the  baffles 
(with  proper  notification  to  sonar);  the  ASWOC  may  quickly  change  to  use 
of  the  helicopter  instead  of  attacking  with  the  ship  or  decide  to  use  a  different 
weapon.  While  the  sequence  of  actions,  and  the  consequences,  are  sometimes 
predictable  with  high  probability,  the  situation  can  be  classified  well  toward 
the  emergent  end  of  the  continuum, 

OPERATIONAL  SEQUENCE  DIAGRAM.  In  an  effort  to  graphically  display  team  inter¬ 
actions,  an  Operational  Sequence  Diagram,  or  OSD,  (Kurke,  1961)  was  developed 
and  is  presented  in  Figure  2.  This  OSD  presents  the  analysis  soenario  which 
is  described  elsewhere;  however,  the  following  comments  are  offered  as  an 
interpretative  aid. 

The  scenario  begins  after  an  intelligence  briefing  and  assumes  that 
the  Search  and  Attack  Unit  is  already  formed  and  the  two  ships  are  proceeding 
toward  the  datum  (last  estimated  position  of  the  submarine)  in  formation. 
It  is  further  stipulated  that  the  trainees  are  part  of  the  crew  of  the  sister 
ship  and  that  the  SAU  Commander  is  on  the  other  ship.  This  role  is  forced 
in  order  that  the  mission  can  prooeed  in  an  orderly  way  without  perturbation 
by  the  possibly  extraneous  actions  of  trainees.  Note,  however,  that  the 
helicopter  is  under  the  ccr'rcl  of  the  ASAC  cn  the  trainees’  ship,  and  when 
the  submarine  is  under  Sonar  contact,  full  control  will  revert  to  the  trainees 
during  the  attaok  phase. 

The  OSD  emphasized  communi cations,  sinae  these  are  the  principal  interactions 
between  the  team.  Other  interactions  can  also  take  place  through  NTD3  consoles, 
which  are  also  indicated  on  the  OSD.  NTDS  oonsole  displays  are  not  indicated 
generally  in  the  OSD,  and  one  should  be  aware  that  console  actions  by  one 
team  member  are  available  for  display  on  the  consoles  of  the  other  team  members. 

It  was  desired  to  emphasize  the  flow  of  information  throughout  the  team, 
and  aiding  arrows  were  added  to  the  OSD  for  this  purpose.  It  car.  be  seen 
that  a  single  event  will  often  trigger  a  cascade  of  communications  throughout 
the  network. 

Particularly  significant  actions  have  been  identified  by  means  of  asterisks 
(*);  the  symbols  used  throughout  the  OSD  are  listed  and  are  a  slight  modification 
of  the  symbols  proposed  by  Kurke  (1S61). 
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Figure  2.  Operational  Sequenoe  Diagrass 
C.  Looalize  oontaot. 
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Figure  2.  Operational  Sequenoe  Diagram 

D.  Maneuver  to  attaok  or  bold. 
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Figure  2.  Operational  Sequence  Diagram 

E.  Identify  and  classify;  maneuver  to 
attaok  or  hold. 
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Figure  2.  Operational  Sequenoe  Diagram 
G.  Couiaot  lost  and  regained. 
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As  part  of  the  task  analysis,  most  communications  are  marked  to  indicate 
a  team  member  who  oould  provide  a  prompting  comment  if  a  communication  is 
omitted,  delayed,  or  in  error  i zigzag  arrows  on  the  OSD),  Training  of  team 
skills  oould  inolude  instruction  for  eaoh  team  member  to  provide  suoh  backup 
communication  assistance. 

COMMUNICATION  LINKAGES.  Communications  were  oounted  (acknowledgements  also) 
in  the  segments  of  the  soenario  and  were  aeoumulated  to  form  a  total,  as 
shown  in  Figure  3,  These  oounta,  or  linkages,  show  the  communication  load 
as  it  varies  throughout  the  soenario. 

As  should  be  expeoted,  the  ASWOC  communicated  to  the  greatest  number 
of  individuals,  with  a  consistently  heavy  load  to  the  SAU  Commander  at  the 
assist  ship.  Otherwise,  the  ASAC  oommunicates  most,  as  is  neoessary  to  control 
the  helioopter,  at  least,  until  the  helioopter  is  cleared  out  of  the  way 
to  permit  a  direot  attaok  from  the  ship.  In  general,  the  ASWFCO  has  the 
lightest  communication  load,  however,  this  picks  up  signifioantly  during 
weapon  delivery. 

It  is  particularly  noteworthy  that  the  ASAC  and  ASNFCO  do  not  ordinarily 
communicate  dlreotly  with  each  other.  Therefore,  the  three  team  members 
do  not  form  a  triad,  but  two  meshed  dyads. 

PERFORMANCE  MEASUREMENT  ANALYSIS 

The  Performance  Measurement  System  (PMS)  to  be  developed  must  serve 
three  purposes:  first,  it  must  provide  real-time  information  oonoerning 
the  performance  of  the  t<*am  and  team  members  during  an  exeroise  in  order 
to  support  the  adaptive  training  capabilities  of  the  devloe.  This  will  inolude 
positive  feedbaok  on  oorrect  performance  and  mistakes  or  leas-than  optimal 
actions  in  time  to  allow  these  problems  to  be  rectified.  It  will  also  provide 
data  to  be  used  by  the  devioe  for  determination  of  possible  alterations  in 
the  exeroise  structure.  This  will  require  that  correct  aotions  and  errors 
by  the  team  members  will  be  recognized  as  they  ooour. 

The  seoond  purpose  of  the  PMS  is  to  provide  data  to  be  used  during  post- 
exercise  debriefing  and  team  evaluation.  While  real-time  performance  assessment 
is  not  necessarily  required  for  this  purpose,  the  performance  data  must  be 
available  within  minutes  after  the  conclusion  of  an  exeroise.  Somewhat  more 
comprehensive  information  may  be  needed  for  this  purpose. 

The  third  purpoae  of  the  PMS  is  to  provide  information  for  the  scientist 
performing  research  on  team  training  and  team  performance.  This  kind  of 
performance  data  oan  be  provided  somewhat  later  than  the  others.  To  meet 
this  purpose,  a  comprehensive  and  highly  flexible  performance  measurement 
system  will  be  required.  This  will  probably  inolude  a  requirement  for  an 
off-line  data  analysis  capability  for  much  of  the  communication  data. 
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ANNOUNCEMENTS 


I.  TOTAL  SCENARIO 


Figure  3.  Coottunioatlon  linkages,  oont. 
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To  meet  these  goals,  the  philosophy  for  the  development  of  the  IMS  functional 
specification  was  to  provide  the  most  detailed  analysis  possible,  given  the 
state  of  the  art,  while  providing  the  maximum  possible  flexibility.  This 
will  allow  desired  alterations  to  be  made  easily  in  support  of  changing  training 
and  research  needs.  This  flexibility  is  critlo&lly  important,  especially 
since  one  of  the  major  research  issues  to  be  addressed  by  the  device  involves 
the  nature  of  the  moat  effective  performance  measurement  In  future  automated 
team  training  systems. 

The  three  principal  sources  of  information  consulted  for  Information 
leading  to  JPMS  development  were:  Naval  ASW  tactioal  doctrine,  interviews 
with  subject  matter  experts  (both  in-house  and  Navy  personnel),  and  examinations 
of  recommendations  from  past  efforts.  For  various  reasons,  however,  only 
a  limited  amount  of  directly  applicable  information  oould  be  gleaned  from 
these  sources.  Performance  assessment  during  current  ASW  training  is  usually 
subjective  and  at  a  rather  global  level.  Although  there  are  a  few  rules-of- 
thumb  concerning,  for  example,  the  amount  of  time  oertain  segments  of  an 
exercise  should  require,  little  objective  measurement  is  used  other  than 
for  oheoks  on  the  accuracy  of  calculations  and  plots.  Most  performance  assessment 
Involves  examination  of  decisions  and  taotioa.  Such  assessments  do  not  easily 
lend  themselves  to  automated  performance  measurement  (of.,  Bell,  197 A). 

For  these  reasons,  the  major  portion  of  the  measurement  atruoture  for 
this  device  will  use  measurement  which  is  untested  for  ASW  tasks.  However, 
the  selected  structure  has  proven  to  be  effective  in  the  airorew  training 
environment. 

SEGMENTATION.  The  performance  measurement  uses  a  mission  segmentation  structure 
based  on  required  or  desired  aotiona  by  the  team  members.  Table  2  presents 
an  analysis  of  the  actions  and  oommunioations  whioh  are  important  to  the 
success  of  the  mission  for  the  selected  soe carlo.  These  events  and  oommuni cations 
are  grouped  into  task  segments  with  eaoh  segment  representing  an  individual 
subtask  whioh  must  be  completed  by  the  ASW  team.  The  events  generally  represented 
important  (often  critioal)  pieces  of  information  that  must  be  exchanged  by 
the  team  members  in  order  for  that  subtaak  to  be  accomplished  successfully. 

The  start  and  stop  conditions  for  eaoh  of  the  segments  are  shown  in 
Table  3.  Measurement  of  eaoh  segment  begins  when  the  event  designated  as 
the  start  event  occurs.  Measurement  of  that  segment  continues  until  the 
specified  stop  conditions  have  been  satisfied.  As  eaoh  specified  event  within 
a  segment  takes  plaoe,  a  record  will  be  made  of  the  time  into  the  mission 
at  which  it  happened,  and,  where  appropriate,  an  evaluation  of  the  aocuracy 
of  the  information  oontained  in  the  communications  will  be  recorded.  For 
the  prototype  device,  a  count  of  the  number  of  critical  events  completed 
and  the  number  and  peroent  of  accurate  communications  will  be  used  to  provide 
performance  measurement  information.  Xn  addition,  the  number  of  unclosed 
aegaents  will  be  counted  and  soored. 
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TABLE  3.  START /STOP  LOGICS  FOR  SCENARIO  SEGMENTS* 


i  gut  ant 

Start 

3  tap 

1. 

1.1 

Tiat  ■  03  alnutaa 

2. 

2.1 

2.13 

3* 

3.1 

3.7 

4. 

4.1 

4.1  4-  S  mo  or  4.2 

5. 

5.1 

5.15 

6. 

6.1 

6.3 

7. 

7.1 

7.4 

8. 

8.1 

8.9  +  5  mo 

9. 

9.1 

9.16  5  mo  or  9.17 

10. 

10.1 

10.4  or  10.3  ♦  5  mo 

11. 

11.1 

11.6  and  11.7 

12. 

12.1 

12.3 

13. 

13.1 

13.3  and  13.4 

14. 

14.1 

14.1  and  14.2 

15. 

15.1 

15.9  and  15.11  and  15.12 

16. 

16.1 

16.1  add  16.2 

17. 

17.1 

17.1  and  17.2 

18. 

18.1 

18.5  and  18.7  and  18.8 

19. 

19.1 

19.1  and  19*2 

20. 

20.1 

20.8  and  20*9 

21. 

21.1 

21.3  and  21.2 

22. 

22.1 

22,4  and  22.5 

23. 

23.1 

23.1  +  5  aao 

24. 

24.1 

24.8 

25. 

25.1 

25.2  and  25.3  and  25.6 

26. 

26.1 

26.11 
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It  is  anticipated  that,  during  completion  of  any  given  segment,  a  large 
number  of  comm uni oations  will  take  plaoe  in  addition  to  the  specified  oritioal 
communications.  A  count  of  these  oommuni cations  will  be  taken  also. 

Start  and  stop  times  for  each  segment  will  be  recorded.  It  is  expeoted 
that  many  of  the  measurement  segments  will  be  ocourring  simultaneously. 
For  example,  at  the  oonclusion  of  Segment  1,  a  series  of  interactions  will 
begin  between  the  ASWOC,  ASAC,  and  helicopter.  These  interactions  constitute 
Segment  2.  During  the  same  time  period,  the  ASWOC  will  probably  be  interacting 
with  the  assist  ship,  bridge,  and  plotter  models  (Segment  3).  Accommodation 
must  be  made  in  the  maohine  design  to  have  several  open  measurement  segments 
at  any  given  time  during  operation  of  the  device. 

CANDIDATE  MEASURES.  Measurement  will  be  made  at  three  different  levels: 
system,  team,  and  individual  performance.  Specific  types  of  measures  to 
be  taken  at  each  measurement  level  are  listed  in  Tables  4,5,  and  6. 

System  level  measurement.  System  level  measurement  inoludes  the  record 
of  the  position  and  behavior  of  all  hardware  elements  of  the  ASW  problem. 
This  inoludes  position  and  movement  of  ownahip,  the  sister  ship,  helioopters, 
buoys,  and  the  target  submarine.  These  measures  are  to  be  recorded,  along 
with  a  time  marker,  a  minimum  of  onoe  per  seoond. 

These  measures  will  be  used  to  develop  a  time  history  record  of  the 
overall  actions  of  the  entire  system,  friendly  assets,  and  opposing  assets. 
They  can  be  used  to  determine  whether  the  SAU  is  moving  in  an  appropriate 
manner  to  dose  on  the  datum,  whether  searoh  assets  cuoh  as  helioopters  and 
buoys  are  deployed  In  an  optimal  position,  whether  the  SAU  is  able  to  dose 
to  an  appropriate  firing  position,  and  whether  they  are  able  to  accomplish 
an  effective  weapons  delivery. 

Team  level  measurement.  This  oategory  of  measurement  includes  the  measures 
of  overall  team  effectiveness  shown  in  Table  5.  These  measures  can  be  divided 
into  three  main  suboategorles:  time  measures,  error  measures,  and  ooomunioatiori 
measures. 

Team  measures  involve  the  performance  and  interaction  of  two  or  more 
of  the  three  positions  filled  by  trainees.  Many  of  these  measures  can  be 
calculated  as  a  composite  of  information  obtained  from  system  level  measures 
(e.g.,  water  entry  point  (WEP)  miss  distanoe  is  determined  by  the  target 
submarine  X  and  Y  position,  and  the  WEP  X  and  Y  position). 

Time  measures  are  determined  by  tracking  the  time  history  of  events 
shown  in  the  Segmentation  Table  (Table  2),. 


47 


NAVTRABQUIPCEN  80-C-0095-1 


TABLE  4.  CANDIDATE  MEASURES  OP  STSTEH  PERFORMANCE. 


OWN  SHIP 


X  Position 
Y  Position 
Course 
Speed 


X  Position 
X  Position 
Course 
Speed 


ASSIST  SHIP 


mas 


X  Position 
Y  Position 
Course 
Speed 


X  Position 
Y  Position 

DATOM 


TARQET  SUBMARINE 


X  Position 
Y  Position 


X  Position 
Y  Position 
Course 
Speed 


WEAPON  WATER  ENTRY  POINT 

X  Position 
Y  Position 
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TABLE  5.  CANDIDATE  MEASURES  OF  TEAM  PERFORMANCE. 

Xliffi 

Attack  tin*  I  (Tiao  from  Step  1.1  to  Stop  24.8) 

Attack  time  II  (Tiae  from  Stop  20.1  to  Stop  24.8) 

Buoy  plant  tiae  (Tiae  from  Stop  1.1  to  Stop  5.17) 

Tiae  in  TDA  (Time  froa  Stop  11.1  to  Stop  24.8) 

Tiao  in  Dog  Box  (Tiae  froa  Step  20.1  to  Stop  23.1) 

RBBOBS 


Water  Entry  Point  Miss  Distsnos 
Buoy  Miss  Distanoe 


smmimim 


Total  Nuabor  of  Coaauni cations 
Maun  Length  of  Coaaunioations  (Soo.) 

Moan  Mosaago  Latonoy  (F*oa  time  of  reportable  event  until  report) 
Nuabor  of  Assists  (Prompts  by  Another  Team  Member) 
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TABLE  6.  C^IDATE  MEASURES  OF  INDIVIDUAL  PERFORMANCE* 

A3W0C 


im 


Tiae  to  Send  Helo  Orders  (Tin*  fro*  Stop  1.1  to  Stop  2.1) 

Time  to  Ordor  Search  Aroe  (Tiae  fro*  Stop  3c?  to  Stop  4.1) 
Zigzag  Patters  Order  Tiae  (Tiae  fro*  Stop  1.1  to  Stop  6.3) 
Tiae  to  Obtain  Weapons  Polioy  (Tiae  fro*  Stop  1*1  to  Step  7.2) 
Firing  Tiae  (Tiae  froa  Step  23*1  to  Stop  24.7) 

errors  ; 

Time  oontaot  ia  in  baffles 
Inoorreot  weapons  selection 

Total  Number 
Mean  Length 
Number  Inoorreot 
Number  Missing 
Number  Late 
Mean  Latenoy 

ASAC 

TIME 

Buoy  Drop  Tiae  (Time  from  Step  1.1  to  Step  2.13) 

Time  to  Report  MAD  Contaot  (Time  froa  Stop  15.1  to  Step  15.3) 
Time  to  Clear  Helo  (Time  from  Step  20.6  to  Step  23*1) 

ERRORS 

Buoy  Position  Error 
MADVEC  Course  Error 
Soram  Course  Error 

CQMMPNIQAXIQH& 

Total  Number 
Mean  Length 
Number  Inoorreot 
Number  Missing 
Number  Late 
Mean  Latenoy 
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TABLE  6.  CANDIDATE  MEASURES  OF  INDIVIDUAL  PERFORMANCE,  CONT. 

ASVFCO 


TIME 


Searoh  Arc  Assignment  (Time  from  Step  9.16  to  Step  10.1) 
WEP  Report  Time  (Time  from  Step  25.1  to  Step  26.1) 

Shot  Status  Report  Time  (Time  from  Step  25.1  to  Step  26.7) 

BKRORS 

Searoh  Arc  Error 
Firing  Bearing  Error 
WEP  Error 

COMMUNICATIONS 

Total  Number 
Mean  Length 
Number  Inoorreot 
Number  Missing 
Number  Late 
Mean  Latency 
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Individual  level  measurement.  This  category  includes  measures  of  performance 
for  each  of  the  three  individual  positions  (ASVOC,  AJ5AC,  and  ASWFCO).  Again, 
the  performance  measures  for  these  individuals  will  include  measures  of  times, 
errors,  and  communications. 

Measures  of  time  will  oonsist  mainly  of  the  time  required  to  process 
a  unit  of  information  reoeived  from  another  team  member  and  to  perform  a 
required  action  or  communication  uaing  that  information.  These  measures 
will  be  obtained  by  tracking  the  time  history  of  events  shown  on  the  Segmentation 
Table  (Table  2).  Measures  of  error  will  be  obtained  by  examining  all  communi¬ 
cations  shown  on  the  Segmentation  Table  to  unoover,  tally,  and  evaluate  missing, 
imocurate,  or  untimely  communications. 

Communication  measures  wixl  include  oounta  and  lengths  of  oommuni cations 
during  individual  segments  and  during  the  exercise  as  a  whole. 

It  should  be  emphasized  that  the  variables  specified  for  measurement 
in  Tables  M,  5,  and  6  apply  specifically  to  the  soenario  used  here  whioh 
involves  two  ships,  one  submarine,  and  an  unarmed  helioopter.  If  obangea, 
additions,  or  deletions  are  made  to  the  assets  Involved  in  the  soenario, 
corresponding  additions  or  deletions  will  have  to  be  made  in  the  list  of 
measirement  variables.  For  example,  position,  course,  speed,  and  time  information 
will  be  needed  for  every  air  and  sea  asset  in  the  exercise. 

In  addition,  the  measurement  system  will  have  to  be  able  to  calculate 
and  accommodate  combinations  of  any  of  these  system  variables  into  more  complex 
measures  defined  by  the  reaearoher  or  user.  For  example,  the  SAU'a  closure 
rate  with  the  submarint  or  ratios  of  incorrect  to  correct  communications 
will  be  needed  in  the  future.  Sufficient  oapability  for  the  real-time  calculation 
of  auob  measures  must  be  provided. 
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SECTION  III 
RESEARCH  EMPHASES 

Specific  options  for  a  research  approach  are  offered  in  this  section 
oased  on  the  literature,  observations  of  current  team  training,  and  an  analysis 
of  ASW  teas  tasks.  The  point  of  view  inplied  here  is  that  the  researoh  use 
of  the  research  tool  oust  be  identified  before  meaningful  hardware  and  software 
specifications  oan  be  made. 

The  broad  research  emphases  selected  are: 

a.  Communications  in  team  training,  for  this  is  a  key  research  area 
(see  Appendix  B) .  These  communications  are  the  team  interactions;  they 
are  distinct  problems  as  observed  in  the  field  and  are  essential  to 
promoting  beneficial  group  phenomena. 

b.  Effective  team  performance  measures,  for  these  are  required  to  identify 
team  phenomena.  They  determine  what  is  accomplished  in  team  training, 
and  are  necessary  for  performance  feedback  (see  Appendix  B) . 

c.  Training  with  synthetic  team  members,  for  the  entire  team  cannot 
be  present.  Modeling  of  synthetic  team  members  is  a  major  step  toward 
understanding  the  underlying  phenomena,  and  such  models  can  be  used 
for  feedback  and  adaptive  interaction. 

d.  Evolutionary  development  of  a  demonstration  system,  for  the  type 
of  system  described  is  a  distinct  advancement  of  the  state-of-the-art, 
and  should  proceed  in  calculated  steps. 

Given  the  above  decisions  about  what  research  to  emphasize,  the  current 
section  attempts  to  derive  the  resulting  implications  for  hardware  and  software. 
A  problem-oriented  organization  is  used:  if  training  is  to  be  accomplished, 
albeit  in  a  research  environment,  (1)  training  objectives  must  be  specified, 
and  the  issues  of  (2)  feedback,  and  (3)  adaptive  variables  must  be  addressed 
with  appropriate  research.  Also,  the  hardware  and  software  must  provide 
for  (4)  the  needs  of  instructors  and  researchers.  Furthermore,  (5)  the  charac¬ 
teristics  of  communication  research,  and  (6)  related  performance  measurement, 
must  be  considered.  And,  finally,  (7)  future  research  possibilities  should 
be  noted  to  indicate  needs  for  flexibility  in  design.  Eaoh  of  these  topi  cs 
is  taken  up  in  order  in  the  following  paragraphs. 

TRAINING  OBJECTIVES 

A  research  tool  for  the  purpose  of  investigating  team  training  issues 
must  necessarily  involve  cases  of  real  training.  This  presents  a  dilemma, 
since  both  the  literature  and  the  training  observation  accomplished  in  this 
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study  point  out  that  specific  team  training  objectives  have  not  been  well 
defined,  and  no  one  seeos  to  be  able  to  artioulate  just  what  it  is  that  is 
accomplished  during  team  training. 

However,  examination  of  the  oolleotive  definitions  of  "group"  and  "team" 
include  the  following  features:  (1)  a  network  of  relevant  oommunioations, 
(2)  a  shared  sense  or  identity,  and  (3)  shared  goal  disposition  with  associated 
norms.  Furthermore,  the  literature  supports  the  conclusion  that  desirable 
group  phenomena  result  from  these  oonditions.  Therefore,  as  a  beginning, 
to  provide  a  basis  for  detailed  exploratory  researoh,  it  is  recommended  that 
these  requisite  conditions  be  adopted  as  training  goals.  In  particular, 
the  flow  of  communications,  understanding  of  mutual  relationships,  and  quanti¬ 
fication  of  specific  team  goals,  are  weaknesses  observable  in  ourrent  team 
training.  Also,  the  Operational  Sequence  Diagrams  (Figure  2)  contain  information 
through  which  team  members  can  prompt  each  other  when  oommunioations  are 
omitted,  delayed,  or  incorrect;  team  training  should  emphasize  suoh  helpful 
actions.  Training  toward  these  ends  should  establish  a  basic  functioning 
team  upon  which  further  research  can  develop. 

The  following  sections  expand  upon  specific  variables  which  can  be  emphasized 
within  the  foregoing  framework. 

FEEDBACK 

The  literature  strongly  supports  the  notion  that  feedback  (or  knowledge 
of  results)  is  an  exceedingly  strong  training  variable,  and  that  researoh 
in  this  area  will  have  a  high  probability  of  payoff.  In  particular,  the 
kind  and  amount  of  individual  and  team  feedback  and  the  immediacy  of  feedbaok, 
requires  heavy  research  emphasis.  It  is  believed  that  comparison  between 
operator  performance  and  that  of  team  member  models  can  be  used  for  automatic 
immediate  feedback,  and  performance  measurement  oan  be  used  to  support  post¬ 
exercise  instructor  debriefing.  In  view  of  the  scenario  analyzed  and  the 
procedure  currently  used  in  operational  team  training,  feedbaok  given  during 
a  training  exercise  and  feedback  given  after  a  training  exercise  requires 
investigation. 

Specific  options  to  be  considered  are: 

a.  Provide  a  prompt  to  the  operator  (ASWOC,  ASAC,  or  ASWFCO)  showing 
the  specific  communications  which  would  be  made  by  the  model  for  that 
station. 

b.  Display  the  model  communications  as  in  (a),  but  only  if:  (1)  human 
operator  communications  are  omitted  or  seriously  delayed;  (2)  operator 
communication  is  in  error  (type  or  oontent);  or  (3)  after  the  operator 
conmunicates,  model  communication  is  displayed  for  verification  to  reinforce, 
as  well  as  to  correct.  It  should  be  noted  that  the  "individual"  who 
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can  provide  a  helping  prompt  (see  the  OSD,  Figure  2)  is  often  one  who 
will  be  modeled  in  the  device  under  consideration;  therefore,  such  prompts 
become  a  form  of  automated  feedback. 

The  following  hardware  features  are  considered  neoessary  to  effectively 
implement  the  above  options: 

a,  A  oolor  Cathode  Ray  Tube  (CRT)  display  with  alphanumeric  and  line¬ 
drawing  graphic  capability. 

b.  An  alerting  mechanism,  including  an  audible  tone,  warning  lamp, 
and  use  of  CRT  oolor. 

o.  Variable  funotion  response  keys  or  touch-panel  array, 
d.  Computer- generated  speeoh  messages. 

ADAPTIVE  VARIABLES 

The  ASW  problem  lends  itself  to  the  manipulation  of  adaptive  training 
variables  of  a  number  ot  different  types.  Some  of  the  more  interesting  adaptive 
variables  are: 

a.  Augmenting  the  trainee's  displays  with  prompting  information  generated 
by  performance  measurement  or  the  model  of  the  trainee's  position. 

b.  Manipulating  positive  or  negative  features  of  team  behavior  of  trainee 
models  to  provide  degrees  of  difficulty  in  the  team  environment. 

c.  Increasing  problem  (setup)  difficulty.  Based  on  the  analysis  performed 
in  this  study,  the  following  sequenoe  is  reoommended  for  initial  researoh: 
(1)  single  ship,  (2)  single  +  one  aircraft,  (3)  dual  ship  +  one  airoraft, 
trainees  in  the  assist  ship,  (t)  dual  ship  +  one  aircraft,  trainees 
and  the  Search  Attack  Unit  Commander  (SAUC),  (5)  dual  ship  +  2  airoraft, 
SAUC,  and  (6)  dual  ship  +  2  airoraft,  SAUC,  smarter  adversary.  Within 
this  sequenoe,  difficulty  can  be  modified  by  increasing  the  required 
level  of  maneuvering. 

d.  Providing  step-wise  or  non-real-time  problem  presentation  (again, 
when  a  single  operator  is  perf orming) . 

e.  Implementing  a  more  intelligent  submarine  adversary. 

While  many  of  the  system  features  required  for  the  above  adaptive  variables 
are  obvious,  the  following  are  listed  for  emphasis: 

a.  Flexible  team  member  and  submarine  models,  anticipating  possible 
future  modifications  or  expansions. 
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b.  Problem  freeze  under  control  of  an  automatic  perform anoe  monitor, 
with  restart  at  any  designated  station. 

o.  Either  interactive  or  automatic  problem  setup  for  quiok  and  easy 
changes  in  problem  diffioulty. 

INSTRUCTOR/ RESEARCHER  CONSIDERATIONS 

It  should  be  olear  that  a  research  tool  should  be  designed  with  the 
needs  of  the  researober  in  mind;  and,  further,  sinoe  training  researoh  will 
also  involve  an  instruotor,  the  needs  of  the  instructor  must  also  be  considered 
(without  exploding  costs  with  the  inolusion  of  a  fully  integrated  instruotor 
work  station).  To  some  extent,  the  design  may  simultaneously  satisfy  the 
needs  of  instruotor  and  researoher;  and  also,  researoh  interaction  will  be 
primarily  off-line  after  the  end  of  training  exeroiaes. 

Features  needed  for  the  instructor/ researoher  inolude: 

a.  Interactive  problem  setup. 

b.  Problem  oontrol— start,  freeze,  oontinue,  stop. 

c.  Comprehensive  plots  of  track  histories,  time  listings  of  events 
and  communications,  and  seleoted  performance  measures. 

d.  Audio  monitoring  capability  for  observers  sinoe  oommunioations  are 
divided  over  a  number  of  ohannels. 

e.  Standard  format  records  for  off-line  analysis,  transportable  media 
(magnetio  tape),  and  convenient  summary  data  analysis  for  data  quality 
monitoring. 

COMMUNICATIONS 

A  major  researoh  and  training  area  is  that  of  team  oommunioations. 
There  is  a  need  for  a  means  of  detailed  analysis  of  oommunioations  occurring 
during  simulated  ASV  exercises.  This  analysis  oan  strive  for  two  goals: 
(a)  the  formulation  of  measurement  illuminating  the  nature  of  team  interactions, 
and  (b)  a  basis  for  establishing  standardized  communication  where  there  is 
none  present. 

Given  a  capability  for  automatic  recognition  of  speeoh,  the  onoe  impossible 
task  of  detailed  measurement  of  volumes  of  speeoh  data  beoomes  feasible. 
The  multitude  of  potential  communication  measures  oan  be  automatically,  or 
scmi-automatioally,  prooessed  to  generate  large  sets  of  oandidate  measures 
which  oan  be  subjected  to  statistioal  selection  teohniques.  In  this  way, 
the  task  of  identifying  new  and  meaningful  measures  of  team  interaction  beoomes 
a  goal  within  reason. 
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While  ASW  communications  to  destinations  outside  the  Combat  Information 
Center  are  standardized  and  formal,  the  equivalent  standardization  does  not 
exist  for  oommunioations  within  the  CIC.  It  oan  be  hypothesized  that  suoh 
standardization  oan  greatly  enhanoe  team  interaction.  Again,  a  capability 
for  speeoh  recognition  can  provide  the  tool  sorely  needed  for  methodically 
analyzing  and  standardizing  the  internal  comauni cations. 

Hardware  features  to  support  the  above  oommunioations  research  include 
the  following: 

a.  Multi-ohannel  audio  recording. 

b.  Speeoh  recognition/understanding  for  three  talkers. 

c.  Off-line  (sami-automatic)  speech  data  reduction  for  (1)  qulok  end- 

of-run  analysis  and  (2)  extended  end-of-day  analysis. 

OFF-LINE  ANALYSIS  OF  COMMUNICATIONS  DATA 

The  envisioned  team  training  researoh  will  provide  an  excellent  environnent 
in  which  to  investigate  the  oommunioation  behavior  and  performance  of  real- 
world  teams  during  the  performance  of  oomplex  tasks.  In-depth  oommunioation 
studies  of  this  kind  are  likely  to  suggest  ways  that  the  effioienoy  of  the 
team  interactions  may  be  improved,  as  well  as  suggesting  areas  for  increased 
n  or  altered  training  emphasis.  This  suggested  use  of  the  research  device 

*  will  require  the  capability  for  a  detailed  linguistio  analysis  of  the  content 

of  all  oommunioations  between  team  members. 

The  process  of  performing  a  linguistio  analysis  of  oral  oommunioations 
is  not  simple.  The  primary  reason  is  that  natural  oral  oommunioation  is 
exceptionally  complex  and  unruly.  In  contrast  to  the  kind  of  carefully  "laun¬ 
dered,  sanitized,  starched,  and  pressed"  prose  usually  studied  by  linguists 
(Chapanis,  1971) >  real  oral  oommunioation  oonsists  largely  of  utteranoes 
which  are,  in  full  or  in  part,  ungrammatical,  incomplete  sentences  or  ideas, 
unintelligible  phrases,  and  non-words. 

Some  problems  in  the  analysis  of  oral  communications  are  described  by 
Chapanis,  Parrish,  Ochsman,  and  Weeks  (1977).  One  kind  of  problem  involved 
words  which  received  a  non-standard  pronunciation,  due  either  to  the  speaker's 
regional  aocent  or  to  an  actual  mispronunciation  of  the  word.  Another  class 
of  problem  involved  partial  or  incomplete  words  such  as  "em,"  "cuz,"  or  "uh." 
Colloquial  or  slang  words  presented  another  difficulty.  This  included  words 
such  as  "yup,"  "nope,"  and  "Jobbers."  Contractions  such  as  "gonna,"  "d'ya," 
and  "gotcha"  were  another  source  of  analysis  difficulty.  In.  addition,  frequent 
interjections  suoh  as  "hmmm,"  "ugh,"  and  "whew"  were  found  to  be  a  oommon 
part  of  the  normal  communications.  Finally,  totally  unintelligible  words 
were  not  unoommon,  although  these  oould  usually  be  identified  by  the  context 
in  which  they  ocourred. 
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In  addition  to  the  irregularities  in  the  individual  words  found  in  normal 
oral  communication,  word  strings  (they  can  scaroely  be  described  as  sentences) 
follow  few  grammatical  rules,  and,  taken  individually,  often  make  little 
sense.  Figure  4  presents  a  quotation  fra m  a  communication  prctoool  discussed 
by  Chapanis  (1976)  and  demonstrates  thiu  point. 

Although  the  verbal  exchange  shown  on  Figure  4  appears  strange  and  awkward 
when  read,  it  is  important  to  note  that  these  spoken  words  apparently  made 
perfeot  sense  to  the  team  members  who  exchanged  them.  The  required  information 
was  effectively  transferred,  and  the  problem  was  correctly  solved. 

Because  of  the  word  recognition  problems  and  sentsnoe  structure  unruliness 
described  above,  any  attempts  at  automatic  linguistic  analysis  are  far  beyond 
the  current  state-of-the-art.  This  suggests  that  a  capability  must  be  provided 
which  will  permit  an  off-line  linguistic  analysis  of  the  communication  oontents. 

This  requirement  can  be  met  by  the  inclusion  of  a  multi-ohannel  tape 
reoorder  configured  such  that  eaoh  ohannel  is  dedicated  to  one  of  the  team 
members.  The  training  sessions  will  be  aoourately  reoorded  as  they  are  taking 
plaoe.  After  the  oonolusion  of  the  session,  the  tapes  will  be  transcribed 
into  hard  copy  and  independently  verified.  The  text  of  the  training  sessions, 
reproduced  in  this  manner,  can  then  be  entered  into  magnetic  tape  or  disk 
storage  for  oomputer  analysis.  Some  of  the  communication  measures  whloh 
could  be  subsequently  derived  are  (a)  total  number  of  words,  (b)  total  number 
of  unique  words,  (c)  type/token  ratios,  (d)  total  number  of  messages,  (e) 
mean  and  variance  of  message  length,  (f)  use  of  different  parts  of  speeoh, 
(g)  coefficient  of  variation  in  words,  messages,  and  (h)  commonality  of  word 
usage  among  teams. 

FUTURE  POSSIBILITIES 

While  the  current  study  is  examining  a  speoiflc  selected  application, 
a  number  of  related  problem  areas  have  been  apparent.  Sinoe  a  tool  is  often 
used  for  purposes  other  than  those  originally  intended,  it  is  believed  appropriate 
to  give  some  design  consideration  to  future  possibilities. 

It  has  been  observed  that  much  of  ourrent  team  training  is  devoted  to 
training  tactics.  Although  tactical  decision  making  is  not  a  team  task  but 
is  the  responsibility  of  the  ASWOC,  the  training  of  ASW  tactics  is  an  intriguing 
and  challenging  subject,  and  perhaps  this  research  tool  could  also  find  appli¬ 
cation  in  this  important  area. 

Similarly,  the  investigation  of  team  training  may  point  the  way  to  the 
design  pf  a  number  of  oost-effeotive  part- task  trainers.  The  ourrent  tool 
could  also  find  application  in  such  design  efforts. 

The  current  Navy  jobs  and  task  assignments  have  been  taken  as  a  given 
in  the  current  study,  but  it  is  not  at  all  certain  that  the  current  jobs 
are  optimally  defined.  The  research  tool  as  presently  envisioned  would  lend 
itself  readily  to  the  re-allooation  of  funotions  between  team  members. 
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Subject  A: 
Subject  S: 
Subject  A: 

Subject  B: 
Subject  A: 


Okay,  I  got  you  but  it’ a  wider  than  it  right? 

Yeah,  a  little  bit  wider. 

A  little  bit  wider... Okay  wait  a  seoond.  Oh  how  wide  is  the 

socket  is  it  you  know  the  socket  itself  is  it 

Okay 

wide  or  thin  ouz  I  h -ve  uh  two  two  just  about  like  that  what 
you  described. 


Figure  4.  Portion  of  oral  communication  protocol  from 

cooperation  problem  solving  study  (Chapanis,  1976). 


The  above  are  only  exemplary:  it  should  be  easy  to  greatly  expand  the 
growth  potential  for  the  present  research  tool.  And,  suoh  consideration 
of  potential  growth  could  lead  to  a  better  design  specification. 
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SECTION  IV 
SYSTEM  CONCEPT 

The  purpose  of  this  section  is  to  present  a  system  conoept  far  the  proposed 
team  training  demonstration  system:  a  general  pioture  of  what  the  demonstration 
system  wil 1  look  like,  what  kinds  of  hardware  it  will  use,  and  what  it  will 
do.  A  well-foraulated  system  conoept  will  serve  in  a  variety  of  capacities 
to  aid  in  the  further  design,  development,  and  refinement  of  the  original 
basic  notion. 

The  approach  in  this  section  i3,  first  of  all,  to  answer  the  question 
"What  will  the  system  look  like?,"  and  give  a  general  description  of  the 
hardware  suite.  Secondly,  an  overall  view  of  the  functional  capabilities 
of  the  system  is  offered  to  answer  the  question  "What  will  the  system  do?" 

WHAT  WILL  THE  SYSTEM  LOOK  LIKE? 

An  observer  who  comes  upon  the  completed  team  training  demonstration 
-system  will  see  a  system  which  is  a  combination  of  a  laboratory  and  of  a 
CIO  slioe-of-life.  There  are  up^o  five  human  participants  in  this  system: 
three  "students"  (ASWOC,  ASWFCO,  ASAC),  one  "instructor,"  and  the  researcher. 
The  "instructor"  and  researoher  may,  in  fact,  be  a  single  individual.  For 
the  initial  look  at  the  functioning  system,  assume  that  all  these  humans 
are  present,  * 

Before  the  students  arrive,  the  instructor  and  researcher  cooperate 
to  set  up  the  ASW  problem  to  be  used  that  day.  Starting  with  a  skeletal 
soenario  and  aided  by  computer  prompts  and  sets  of  menus,  the  instructor 
and  researcher  proceed  to  assign  initial  positions  and  speeds  to  the  players, 
to  set  up  the  environment  (ocean  conditions,  visibility,  wind,  weather), 
and  to  establish  the  weapons  and  motion  capabilities  of  all  craft  in  the 
exercise.  The  researcher,  ohoosing  from  a  oomputer- presented  menu,  assigns 
and  sets  up  performance  measures  to  be  used  during  the  problem  and  selects 
data  to  be  recorded  by  the  computer  during  the  exercise. 

Sometime  during  this  set-up  process,  the  students  arrive  and  sit  down 
at  their  consoles.  The  physical  arrangement  they  see  is  a  pattern  of  three 
student  consoles  sitting  at  the  vertices  of  a  roughly  equilateral  triangle, 
facing  outward,  while  the  instructor/researcher  station  sits  at  the  center 
of  this  triangle.  (See  Figure  5)  The  consoles  the  students  see  are  not 
operational  consoles,  but  functional  facsimiles.  Should  the  observer  look 
over  the  ASWOC* s  shoulder,  for  example,  the  console  picture  he  would  see 
might  look  like  Figure  6.  Although  their  physical  appearance  is  identical 
from  a  functional  point  of  view,  the  consoles  are  individually  tailored  for 
each  team  member's  role.  The  ASWOC's  console  mimics  an  OJ197  console,  while 
the  ASWFCO* s  and  the  ASAC's  consoles  mimic  an  0J19M  console  operating  in 
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the  weapon  and  the  ASAC  modes,  respectively.  Each  console  also  contains 
a  standard  keyboard  to  permit  entry  of  numeric  data  and  to  allow  each  student 
to  sign  on. 

Sinoe  the  set  of  students  have  not  used  the  system  before,  they  sign 
on  individually  and  begin  to  receive  instruction  from  the  computer  as  to 
how  the  system  works  and  how  to  get  full  use  of  their  consoles.  As  part 
of  the  pnooesa  of  familiarization,  the  students  are  told  of  the  speeoh  recognition 
and  generation  systems  and  their  limitations.  The  computer  oolleots  voice 
reference  data  from  eaoh  student  and  stores  the  voice  reference  patterns 
for  later  use. 

Once  everyone  is  ready  and  problem  set  up  is  oomplete,  the  system  gives 
the  students  an  intelligence  briefing  which  explains  the  state  of  their  taotioal 
world  as  they  take  over,  including  information  about  a  possible  submarine 
in  the  area  and  its  rough  looation  from  an  intelligence  report.  This  briefing 
is  intended  to  Inform  the  students  about  the  capabilities  of  their  ship, 
the  oharaoter  of  the  air  support,  the  hypothesized  hostile  submarine  type, 
and  various  other  environmental  information. 

When  the  briefing  is  oomplete,  the  problem  begins.  Motion  of  all  oraft 
in  the  exeroiae  begins,  and  the  team  members  begin  to  respond  to  this  simulated 
ASW  environment  as  if  it  were  the  real  thing  -  the  game  is  afoot  I  The  full 
ASW  team  -  ownship,  assist  ship,  and  supporting  airoraft  -  pursue  their  roles 
in  attempting  to  looallze  the  submarine.  During  this  activity,  the  team 
members  apeak  to  eaoh  other  and  to  all  the  other  simulated  support  personnel 
as  needed,  and,  at  appropriate  times,  the  simulated  support  personnel  respond 
to  team  member  oommunioatlons  and  Initiate  communications  of  their  own. 

Sometime  early  on  in  the  problem,  sonar  contact  with  a  possible  submarine 
is  reported  and  additional  contacts  are  sought  to  establish  the  submarine's 
oourse.  As  the  paoe  picks  up,  ownship  moves  in  on  the  submarine,  fires  a 
weapon,  and  retires  to  a  safe  range. 

As  all  this  activity  is  going  on,  the  instructor  occasionally  moves 
around  to  watch  over  team  members’  shoulders  to  see  the  problem  as  they  see 
it.  Baok  at  his  console,  the  instructor  can  see  the  whole  picture:  positions 
of  all  friendly  oraft  as  well  as  the  position  and  the  path  of  the  submarine, 
as  shown  in  Figure  7. 

When  the  problem  is  concluded,  the  team  members  ar-  debriefed  individually 
by  the  oomputer  at  their  oonsoles  and  then  collective  y  by  the  instructor. 
As  an  aid  for  this  collective  debriefing,  the  instructor  has  available  a 
oomT"  ter-created  plot  of  the  motion  of  all  craft  in  the  exercise,  annotated 
to  show  positions  at  crucial  times  during  the  problem. 

When  the  problem  is  oomplete,  the  researcher  may  consult  with  the  instructor, 
comparing  observations  and  computer-generated  evaluation  with  the  instructor's 
own  assessment.  The  researcher  can  request  a  hard  copy  listing  of  data  the 
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computer  gathered  during  the  problem  and  may  store  other  information  on  magnetic 
tape.  He  also  has  available  to  him  an  event-marked  tape  recording  of  all 
verbal  communications  which  occurred  during  the  problem. 

WHAT  EQUIPMENT  IS  NEEDED? 

A  team  training  demonstration  system  like  the  one  just  described  needs 
computers,  speech  recognition  and  generation  equipment,  system  CRTs,  display 
consoles,  mass  data  storage,  a  line  printer,  and  a  multi-channel  tape  reoorder. 
A  potential  system  configuration  is  sketched  in  Figure  8.  The  Individual 
components  are: 

a.  Four  medium-sized  minicomputers  on  the  order  of  a  Data  General 
Eclipse  S-230  or  a  DEC  PDP-11/ 24. 

b.  Two  or  three  NEC  DP-200  connected  speech  recognition  systems. 
This  NEC  device  is  so  new  that  its  specifications  are  not  yet  avail¬ 
able.  The  older  DP-100  can  handle  two  channels  -  henoe  two  speakers 
-  simultaneously,  but  it  is  not  known  yet  if  the  DP-200  will  oome 
with  a  multi-channel  capability.  Should  a  new  oonneoted  speeoh 
device  appear  on  the  market  soon,  it  certainly  would  be  investigated 
carefully  for  its  applicability  to  the  team  training  problem. 

c.  A  text-to-speech  voice  generation  system.  Votrax  has  reoently 
marketed  such  a  device,  and  others  may  soon  follow  suit. 

d.  A  speech  replay  system  designed  to  produoe  speeoh  from  digital 
data  stored  on  disk.  This  facility  would  be  instrumental  in  giving 
individualized  voices  to  the  simulated  ASW  support  personnel. 
Quite  reoently,  a  number  of  systems  have  become  available  whioh 
claim  to  support  good  quality  reproduction  of  digitally  reoorded 
speeoh  at  reasonable  bit  rates.  It  would  be  desirable  for  this 
device  to  have  a  recording  capability  as  well  to  simplify  the  speech 
enooding  process. 

e.  A  multi-channel  tape  recorder  capable  of  recording  8  to  16  tracks 
of  speeoh,  capable  of  operating  under  computer  control,  and  capable 
of  putting  event  markers  on  tape  on  computer  command.  The  ability 
to  record  a  reasonably  large  number  of  tracks  is  needed  so  that 
each  trainee's  speech  and  each  model’s  speech  can  be  recorded  separ¬ 
ately.  The  instructor  may  also  wish  to  record  comments  for  later 
replay  or  use  by  the  researcher. 

f.  A  line  printer  capable  of  supporting  at  least  a  medium  resolution 
plotting  package. 

g.  Four  system  CRTs  to  support  development  and  maintenance  of  the 
system. 

h.  Disk  unit  for  each  computer,  capable  of  storing  at  least  50  Megabytes. 

i.  Magnetic  tape  unit  to  support  system  development,  and  to  facilitate 
storage  of  significant  amounts  of  research  data  and  training  reoords. 

j.  Three  student  consoles  with  basically  the  same  physical  appearance. 
Eaoh  console  consists  of  one  raster-scan  display  unit,  a  keyboard, 
a  communication  panel  for  microphone  plugs,  volume  oontrol  knob, 
voice  level  meter,  and  circuit  switches;  as  well  as  a  capacitor  touch- 
button  panel,  similar  to  that  shown  previously  in  Figure  6.  The 
oonsole  must  also  have  a  track  ball  and  track  ball  related  function 
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Figure  8.  Basic  system  architecture. 
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keys.  One  of  the  three  display  units  (the  ASWOC's)  should  be  a 
color  unit.  The  other  two  can  be  monochromatic. 

k.  One  instructor/ researcher  console  having  the  same  basic  shape  as 
the  student  consoles  but  with  the  screen  tilted  more  toward  the 
horizontal,  This  console  should  provide  a  capability  for  the  instructor 
or  researcher  to  monitor  any  communication  circuit,  and  to  reproduce 
at  his  own  display  the  entire  contents  of  any  one  of  the  team  member's 
displays. 

The  equipment  suite  described  here  represents  an  estimate  of  hardware  required 
to  support  the  entire  system  concept.  A  section  VII  of  this  report  addresses 
the  possibility  of  a  staged  approach  to  system  construction.  The  full  system 
as  described  in  this  section  would  be  the  end  product  of  the  Stage  3  implemen¬ 
tation.  The  subsets  of  equipment  needed  to  support  Stages  1  and  2  are  sketched 
in  Figures  9  and  10,  respectively. 

WHAT  WILL  THE  SYSTEM  DO? 

Having  described  the  team  training  demonstration  system  from  the  perspective 
of  an  observer,  consider  now  the  internal  processing  which  is  required  to 
support  system  operation.  In  the  first  place,  a  number  of  ASW  support  personnel 
must  be  modeled,  including  the  SAU  commander,  assist  ship  personnel,  air 
support  persons,  sonar  officer,  bridge,  surface  watch  officer,  and  plotters. 
The  environmental  simulations  include  the  basic  physical  environment,  motion 
models,  waapons,  sonar,  radar,  magnetic  anomaly  detection,  and  visual  detection. 
A  model  of  the  enemy  will  simulate  the  hostile  unit's  behavior,  capabilities, 
and  motion. 

In  addition  to  this  bulk  of  basically  supporting  simulation,  team  member 
simulation  is  required.  The  team  training  demonstration  system  must  be  capable 
of  operating  when  any  (or  all)  of  the  human  team  members  are  absent.  Specific 
requirements  which  arise  from  this  are  desoribed  in  Section  VI,  System  Require¬ 
ments.  This  point  implies  that,  there  is  a  computer  model  of  each  team  member 
which  is  oapable  of  operating  in  place  of  its  human  counterpart.  The  team 
members  can  ser’fe  three  additional  functions:  to  provide  information  for 
online  feedback  to  the  human  team  member,  to  create  a  yardstick  for  performance 
measurement  to  use  in  evaluating  the  human  team  member's  action,  and  as  an 
aid  for  the  automated  speech  understanding  system  to  provide  reasonable  predi¬ 
ctions  about  what  a  given  team  member  might  say  next. 

Next,  communications  capabilities  are  required.  The  system  conoept 
is  centered  around  communications  among  team  members  and  between  individual 
team  members  and  simulated  support  personnel.  A  significant  part  of  this 
communication  is  verbal,  so  the  system  must  have  the  ability  to  understand 
what  is  spoken  by  the  team  members,  to  act  based  upon  that  understanding, 
and  to  generate  simulated  verbal  responses  or  initiate  verbal  communications 
as  necessary  to  support  the  total  simulation.  More  broadly,  the  system  must 
monitor  verbal  or  cousole-to-con30le  communi cations  between  any  team  member 
and  any  real  or  simulated  target  of  the  communication  effort.  Furthermore, 
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the  system  must  simulate  appropriate  responses  to  these  communications  and 
initiate  coamuni cations  when  that  is  appropriate.  Finally,  to  aid  the  analysis 
of  communications  after  the  fact  by  the  researcher,  the  system  must  maintain 
annotated  records  of  all  consol  e-to-console  and  understood  verbal  ooma  uni  cations 
as  well  as  record  regular  time  tags  on  one  channel  of  the  audio  tape  so  that 
the  system's  reoords  oan  be  correlated  with  the  audio  recording. 

The  system  must  be  capable  of  a  limited  kind  of  active  instruction  - 
enough  to  teach  the  team  members  how  to  use  the  consoles,  how  to  handle  the 
communications  system,  how  to  facilitate  speech  recognition,  and  how  to  ask 
for  help.  This  capability  will  also  be  employed  to  give  detailed  instructions 
to  the  team  members  during  reference  pattern  collection  and  validation. 

A  sophisticated  and  flexible  performance  measurement  system  is  required 
which  is  oapable  of  trying  out  many  different  varieties  of  candidate  measures 
for  effective  team  performance.  To  assess  each  individual  team  member's 
contribution  to  overall  team  performance,  the  performance  measurement  subsystems 
will  traok  each  individual's  actions  and  evaluate  individual  strengths  and 
weaknesses.  Slnoe  much  of  the  effort  in  building  good  performance  measures 
will  be  based  on  data  gathered  by  the  system,  the  system  will  provide  detailed 
reoord  keeping  and  will  preserve  data  in  standard  formats  for  use  by  other 
off-line  analysis  and  data  reduction  programs. 

Feedback  is  an  important  issue  for  team  training,  and  this  system  will 
offer  a  variety  of  means  for  providing  feedback.  The  ASWOC’s  console,  for 
example,  allows  the  researoher  to  explore  the  use  of  color  graphios  to  provide 
feedback  and  information  beyond  that  which  an  ordinary  operational  console 
provides.  Furthermore,  the  system  must  be  capable  of  providing  effective 
positive  feedbaok,  a  capability  which  is  absent  in  most  automated  training 
systems.  The  capabilities  furnished  by  sophisticated  team  member  models 
make  the  process  of  providing  feedbaok  -  positive,  qualified,  or  negative 
-  much  easier. 

(The  careful  reader  will  notice  a  funotion  in  Figure  6  denoted  "Sigh!" 
Why  "Sigh!"?  The  apparently  facetious  oonsole  function  named  Sighl  is  not 
at  all  intended  to  be  flippant.  Instead,  it  should  provide  a  kind  of  pressure- 
release  valve  for  the  trainee  when  he  finds  that  he  is  fighting  the  system, 
being  misunderstood  by  the  speech  misunderstanding  software,  or  generally 
having  a  bad  time.  The  funotion  of  Sigh!  is  to  offer  an  opening  for  student 
initiative  in  giving  feedback  to  the  system:  "You  don't  understand  me," 
"you  don’t  respond  to  what  I  say,"  or  "you're  driving  me  orazy."  Our  experience 
with  the  automated  training  systems  leads  us  to  believe  that  the  trainee 
occasionally  beoomes  very  frustrated  and  often  has  no  acceptable  way  to  deal 
with  the  frustration.  The  Sigh!  function  lets  the  trainee  talk  back.  Of 
oourse,  "Sigh!"  is  not  intended  to  be  the  sole  interactive  mode  for  the  system. 
The  function  of  "Sigh!"  is  to  offer  the  user  a  capability  for  quick  reaotion 
to  the  system  while  an  exercise  ip  in  progress,  and  to  initiate  detailed 
recording  of  the  circumstanoes  which  produced  the  user's  reaction.  When 
just  one  team  member  is  involved,  the  Sigh!  funotion  oould  freeze  the  exercise 


71 


NAVTRAEQUIPCEN  80-C-0 095-1 


and  initiate  a  more  prolonged  interactive  sequence.  In  this  event,  the  Sigh  I 
function  would  aot  principally  as  a  device  to  freeze  the  problem  action  and 
to  call  up  a  full-scale  menu  for  interaction.) 

The  system  must  also  allow  for  oertain  kinds  of  controlled  team  training 
experimentation.  For  example,  the  influence  of  one  particularly  weak  team 
member  could  be  explored  fairly  carefully.  Using  an  ASAC  model  with  controlled 
pieoes  of  misinformation  or  lack  of  knowledge  can  allow  the  researcher  to 
see  how  team  performance  degrades  as  one  individual's  (the  ASAC's)  performance 
degrades.  This  system  will  also  provide  the  facility  to  support  a  weak  human 
team  member  by  providing  him  with  individually  tailored  prompting  messages. 
In  this  way,  team  performance  can  be  observed  while  a  weak  team  member  is 
being  compensated  for. 

The  team  training  demonstration  system  will  also  provide  the  capability 
to  explore  the  adaptive  component  of  training  in  a  variety  of  ways.  The 
enhancement  or  degradation  of  team  member  performance,  described  above,  oould 
also  serve  as  an  adaptive  variable.  Other  dimensions  of  adaptation  will 
ohange  the  environment.  Some  possibilities  include  modifying  the  sonar  dispersion 
exponent  and  influencing  the  range  at  whioh  sonar  contacts  oan  be  gained; 
modification  of  the  wind  or  sea  state  whioh  affeots  sonar  similarly;  making 
the  submarine  faster  or  slower,  more  or  less  evasive;  increasing  or  decreasing 
the  number  of  supporting  airoraft;  controlling  the  number  and  kinds  of  weapons 
available. 

In  summary,  the  team  training  demonstration  system  will  be  supported 
by  a  simulation  system  with  several  component  subsystems.  It  is  designed 
to  monitor,  aontrol,  and  simulate  communications  between  a  team  member  and 
any  other  real  or  simulated  member  of  the  full  ASW  crew.  In  support  of  this 
communications  control  facility,  there  will  be  a  state-of-the-art  speech 
recognition  system,  a  text- to- speech  voice  generation  system,  a  digital  voioe 
generation  system  coming  from  the  newest  technology,  a  computer-controlled 
multi-channel  tape  reoorder,  and  sophisticated  software  to  use  these  devices 
to  their  fullest  capabilities.  The  system  is  also  designed  to  evaluate  a 
range  of  proposed  performance  measurement  tools,  to  permit  new  kinds  of  per¬ 
formance  measurement  techniques,  and  to  explore  some  of  the  adaptive  components 
of  team  training. 
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SECTION  V 

TECHNOLOGICAL  RESEARCH  ISSUES 

The  team  training  demonstration  system  whose  functional  description 
is  presented  on  these  pages  must  be  considered,  in  the  broad  context  of  training 
system  technologies,  as  a  system  which  has  both  hybrid  and  transitional  aspects. 
The  system  is  hybrid  in  that  both  traditional  and  less  traditional  approaches 
are  oalled  upon,  often  in  the  same  subsystem.  Below  we  deaoribe  a  simulation 
subsystem  which  employs  both  traditional  and  non- traditional  approaches  to 
the  various  simulation  problems.  The  speech  generation  system  design  is 
also  a  hybrid  of  old  and  new:  well  known  technologies  employing  generation 
of  canned  phrases  are  called  for,  as  well  as  newer  techniques  to  construct 
phrases  dynamically,  as  the  need  arises.  Furthermore,  traditional  modes 
of  performance  measurement  will  be  oalled  upon,  yet  the  teas  training  environment 
seems  to  call  for  a  smarter  mode  of  performance  measurement,  like  that  a 
good  human  instructor  could  provide. 

The  system  being  described  is  also  very  muoh  transitional.  It  is  not 
a  traditional  automated  training  system,  nor  is  it  avant-garde.  It  maximizes 
the  exposure  of  some  new  techniques  and  new  technologies  which  show  high 
potential  for  the  team  training  environment,  yet  minimizes  the  teohnioal 
risks  associated  with  the  relatively  new  and  as  yet  unproven.  In  the  sections 
which  follow,  we  discuss  four  areas  of  interest  for  the  development  of  techno¬ 
logical  tools  to  support  team  training  systems.  In  particular,  we  address 
those  specific  technological  Issues  which  this  ASW  team  training  demonstration 
system  should  be  able  to  examine  in  some  detail. 

SIMULATION 

The  team  training  demonstration  system  shall  utilize  traditional  modeling 
techniques  for  all  simulation  subsystems  which  are  not  aonoerned  with  the 
modeling  of  team  member  functions.  In  addition,  the  system  whioh  is  described 
on  these  pages  shall  be  capable  of  modeling  the  functions  of  ASWOC,  ASAC, 
and  ASWFCO  to  a  degree. of  fidelity  sufficient  to  allow  each  team  member  model 
to  substitute,  in  a  training  setting,  for  the  corresponding  missing  human 
team  member.  This  requirement  calls  for  a  qualitatively  more  sophisticated 
approaoh  to  modeling,  inasmuch  as  complex  human  behavior  is  involved  in  a 
dynamically  changing  environment  where  there  is  a  Isrge  range  of  potential 
acceptable  actions. 

The  principal  technological  issues  involved  here  are  how  well  this  can 
be  done,  and  what  techniques  are  appropriate  to  do  it.  Simulations  like 
the  modeling  of  missing  team  members  are  knowledge-intensive,  as  opposed 
to  number  or  data  intensive.  Since  "chunks'*  of  knowledge  are  the  raw  material 
of  this  kind  of  simulation,  means  must  be  found  to  represent  this  knowledge 
adequately,  to  manipulate  it  in  order  to  generate  conclusions  and  decisions 
about  actions  to  be  taken,  and  to  control  the  process  by  which  "old"  knowledge 
is  used  to  generate  "new"  knowledge  and  decisions.,  In  the  past  ten  years , 
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several  systems  designed  to  work  at  the  knowledge  engineering  level  have 
been  built  and  they  show  great  potential.  (A  few  of  them  are  described  later 
in  the  section  dealing  with  system  simulation  design  considerations.)  A 
fundamental  issue  here  is  how  good  a  knowledge-based  system  can  be  built 
which  will  function  in  real-time  on  available  small  scale  computer  systems. 

SPEECH  UNDERSTANDING 

The  speech  understanding  subsystem,  which  is  described  later,  must  handle 
the  problem  of  understanding  connected  speech  for  three  speakers  at  the  same 
time,  in  an  environment  which  is  ordinarily  characterized  by  a  large  number 
of  relatively  unstructured  communications.  An  obvious  initial  approach  to 
this  problem  is  to  enforce  some  stylization  of  speech  (basically  standardization) 
and  to  Inhibit  "unnecessary"  collateral  conversation.  A  difficult  enough 
problem  remains  even  if  this  assumption  is  made.  If  the  mere  natural  unrestrained 
speeoh  of  a  normal  CIC  were  to  be  allowed,  the  speech  understanding  problem 
would  beoome  muoh  more  nevere.  One  significant  reason  why  the  soope  of  the 
ASW  problem  was  restricted  as  it  was  is  because  of  the  present  day  limitations 
of  speeoh  recognition  systems.  To  move  in  the  direction  of  permitting  a 
large  volune  of  relatively  unrestrained  communications,  advanoes  in  recognition 
devioes  and  speech  understanding  software  are  essential.  Uere  the  technology 
available,  the  functional  description  of  the  speech  understanding  subsystem 
would  call  for  a  hybrid  speech  recognition  system  which  was  in  part  a  key 
word  spotter  and  in  part  a  connected  speech  recognizer.  The  key  word  spotter 
would  look  through  incoming  speech  for  key  words  or  phrases.  When  such  a 
word  or  phrase  was  found,  the  connected  speeoh  recognition  prooess  would 
oome  into  play,  recognizing  "relevant"  speech  which  the  word  spotter  had 
located.  In  this  way,  speech  which  was  determined  to  be  signifiount  could 
be  extracted  and  understood,  while  other  speech  could  be  ignored.  This  would 
permit,  in  principle,  a  relatively  unconstrained  speech  environment  and  a 
big  vocabulary. 

Since  hybrid  speech  technologies  like  the  one  just  described  don't  exist 
yet,  perhaps  a  parallel  effort  to  develop  the  technology  is  needed.  It  is 
the  case,  however,  that  a  more  artificial  speech  environment  must  be  tolerated 
if  speech  is  to  be  used  at  all  at  the  present  time.  But  significant  questions 
remain;  How  well  can  speech  understanding  be  done  for  three  people  at  the 
same  time  in  this  more  constrained  environment?  How  might  word  spotting 
techniques  be  employed  to  push  toward  a  less  constrained  environment?  How 
can  other  system  knowledge  sources  be  used  to  aid  the  understanding  prooess, 
particularly  by  enhancing  the  ability  to  predict  what  might  be  said  under 
a  given  set  of  circumstances. 

SPEECH  SYNTHESIS 

Siaoe  techniques  for  speech  synthesis  and  speech  replay  have  been  available 
for  some  years  now,  there  is  a  tendency  to  take  good  synthesized  speech  for 
granted.  To  a  certain  degree,  this  is  fair;  Votrax,  for  example,  is  now 
available  on  a  single  chip  for  a  low  price.  But  good  quality,  human-sounding 
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speech  is  harder  to  get.  Digital  storage  and  reconstruction  of  speech  at 
reasonable  bit  rates  are  possible,  but  the  quality  of  speech  has  thus  far 
been  disappointing,  large  disk  storage  requirements  still  remain,  and  the 
standard  techniques  for  piecing  together  strings  of  individually  recorded 
items  sometimes  produce  unacoeptably  ohoppy  speech.  There  are  chips  available 
with  linear  predictive  coding  (LPC)  encoded  speech  for  replay,  but  the  chips 
require  an  expensive  oustom  fitting  for  specialized  vocabularies. 

In  addition  to  these  problems  just  with  producing  good  quality  speech, 
there  is  another  question.  Suoclnotly  stated:  What  should  be  said? 

To  convey  information  to  a  human  being  as  another  human  being  might, 
the  system  must  call  upon  its  knowledge  of  the  subject  matter,  of  the  intended 
listener,  of  the  importance  of  the  communication,  and  of  the  syntax  of  English 
to  put  the  information  into  words  and  create  fluid  speech.  Processes  whioh 
are  automatic  in  human  beings  as  they  marshall  information  and  choose  words 
to  communicate  must  somehow  be  recreated  or  simulated  by  the  speech  generation 
system  when  it  "deoides”  what  is  to  be  said.  How  it  should  be  said  i3  another 
question.  Most,  if  not  all,  existing  automated  training  systems  whioh  use 
speeoh  generation  rely  on  the  reproduction  of  oanned  phrases  -  phrases  which 
have  previously  been  built  in  digital  form,  or  phrases  whose  constituent 
phonemes  have  been  selected  and  entered  in  text  files.  In  an  environment 
like  the  CIC  during  an  ASH  exercise,  a  large  number  of  event  sequenoes  are 
possible,  and  the  speech  generation  capability  should  accommodate  the  production 
of  phrases  for  all  eventualities.  This  suggests  strongly  that  some  kind 
of  text-to-speech  system  (suoh  as  the  one  recently  introduced  by  Votrax) 
would  be  indispensable  in  supporting  the  voioe  synthesis  aspects  of  simulation. 
To  oreate  phrases  dynamically,  the  simulation  subsystem  must  be  smart  enough 
to  know  what  to  say.  That  is,  it  should  have  the  capability  to  use  the  infor¬ 
mation  available  to  it  in  order  to  make  these  decisions:  what  to  say,  when 
to  say  it,  and  to  whom  to  say  it. 

There  is  also  a  peripheral  issue,  straddling  the  boundary  between  training 
and  technological  research  questions.  Since  there  are  quite  a  number  of 
personnel  in  the  ASH  environment  whose  voices  must  be  simulated,  the  speech 
generation  subsystem  must  produce  different  voices.  How  many  distinct  voices 
are  adequate  for  the  task?  Using  a  single  Votrax  for  all  the  voices  would 
probably  produce  unacceptable  confusion,  but  how  many  like-sounding  voices 
can  be  tolerated? 

PERFORMANCE  MEASUREMENT 

Another  group  of  research  issues  which  are  implicitly  involved  in  the 
design  of  the  teas  training  demonstration  system  concern  performance  measurement 
(PM).  Performance  measurement  is  usually  considered  the  province  of  the 
training  analyst,  but  there  are  several  PM  issues  which  impinge  on  the  area 
of  software  technology. 


75 


N AVTR AEQUIPCEN  80  -000 95-1 


The  scope  of  FM  is  often  taken  to  be  limited" to  the  process  of  monitoring 
events  which  occur  in  the  training  exercise  and  to  the  activity  of  correlating 
collections  of  combinations  of  measurements  taken  with  respeot  to  these  events. 
Certainly  data  of  this  kind  should  have  a  significant  bearing  on  the  evaluation 
of  a  student's  performance*  but  the  impartial  observer  may  feel  that  something 
is  being  missed  in  this  process  and  that*  in  a  oertain  sense*  the  forest 
Is  overlooked  in  favor  of  the  trees.  The  presenoe  in  the  proposed  system 
of  computer  models  of  competent  team  members  offers  an  opportunity  for  evaluation 
of  a  human  team  member’s  performance  from  a  broader  point  of  view.  The  ideal 
of  an  extremely  patient  and  fully  knowledgeable  instructor  could  be  realized 
in  a  knowledge-based  system.  With  such  a  system  one  could  create  a  capability 
for  understanding  something  of  what  a  team  member  is  really  trying  to  do 
and  of  unoovering  the  faulty  conceptual  notions  underlying  less  than  optimal 
team  behavior. 

Another  performance  measurement  issue  has  particularly  significant  rami¬ 
fications  for  software  design  and  system  integration.  How  do  system  designers 
and  programmers  implement  the  performance  measurement  system  which  the  training 
analyst  recommends?  In  the  past  this  has  been  a  sore  point.  Often*  PM  systems 
have  required  ooding  the  same  knowledge  about  a  training  situation  in  a  dozen 
or  more  different  places.  Occasionally,  this  knowledge-ooding  process  is 
not  entirely  consistent,  and  the  inconsistencies  must  be  discovered,  then 
painstakingly  resolved  and  oorreoted.  Furthermore*  beoause  PM  is  usually 
overlayed  on  the  other  subsystems,  nearly  all  the  other  software  must  be 
debugged  before  PM  oan  be  tested.  This  means  that  one  of  the  most  important 
instructional  system  features  is  debugged  only  at  the  end  of  the  testing 
process,  and  rarely  can  enough  time  be  spent  to  see  that  PM  software  lives 
up  to  the  training  analyst's  original  oonoeption.  Then,  too,  there  is  the 
PM  domino  effect;  Failing  one  PM  minitest  early  in  an  exercise  can,  beoause 
of  an  otherwise  quite  reasonable  design,  cascade  to  produce  failures  of  all 
succeeding  PM  minitests,  no  matter  what  the  student  does. 

The  performance  measurement  system  should  really  be  smarter  than  all 
this.  Most  of  the  difficulties  just  described  could  be  overoome  if  the  PM 
system  had  access  to  an  organized  knowledge  base  in  which  knowledge  oould 
be  consistently  and  irredundantly  represented  both  for  system  use  and  for 
appraisal  by  the  training  analyst. 
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SECTION  VI 
SYSTEM  REQUIREMENTS 

An  automated  training  system  is  a  complex  entity  with  many  Interacting 
capabilities.  Figure  12  provides  a  simple  illustration  of  this  point.  It 
suggests  that  there  is  a  oore  of  knowledge  upon  whioh  primary  system  capabilities 
rest,  and  furthermore  that  there  are  many  other  capabilities  which  can  be 
added  to  embellish  the  outward  aspect  of  such  a  system.  Not  all  of  these 
embellishments  are  appropriate  to  add  to  a  researoh  device,  however,  and 
the  following  discussions  describe  which  aspects  are  important  to  the  successful 
demonstration  of  the  concept,  and  specify  their  functional  requirements. 

SIMULATION  REQUIREMENTS 

To  support  an  investigation  of  team  training  issues  in  a  Combat  Information 
Center  involved  in  an  ASW  mission,  a  simulation  of  the  tactical  environment 
as  it  would  be  seen  by  the  team  members  shall  be  provided.  This  simulation 
shall  be  reasonably  faithful  to  real-life,  and  it  shall  also  attend  to  the 
degrees  of  relevance  of  various  environmental  stimuli  as  they  affect  the 
team  members.  For  example,  sonar  data  are  important  elements  in  decisions 
which  the  ASWOC  makes.  But  only  a  processed  version  of  these  data  is  required 
by  him:  he  needs  only  an  indication  of  the  strength  of  the  oontaot,  together 
with  the  bearing  and  the  range,  if  they  are  available.  The  raw  "pings"  are 
essentially  irrelevant  to  him. 

What  is  the  function  of  simulation?  Simulation  is  basically  imitation: 
the  simulation  under  discussion  here  oonsists  of  software  (and,  to  some  extent, 
hardware)  which  imitates  or  mimics  either  something  in  the  real  world  (the 
physics  of  sonar  or  radar  or  magnetic  anomaly  detection;  an  operational  console, 
the  dead-reckoning  table,  etc.)  or  someone  in  the  real  world  (SAU  commander, 
sonar  operator,  helioopter  personnel,  etc.).  Th»  simulation  must  be  directed 
foremost  by  the  functional  requirements  whioh  the  system  plaoes  on  it.  Thus 
the  end  products  of  the  various  simulation  subsystems  are  most  significant 
and  the  processes  which  yield  those  end  products  are  less  important.  The 
fidelity  which  is  required  of  the  various  simulation  subsystems  is  not  an 
easy  issue  to  pin  down,  and  perhaps  it  is  best  approaohed  negatively.  No 
simulation  should  produce  a  result  which  is  substantively  at  odds  with  the 
real  behavior  of  what  is  being  simulated.  Furthermore,  no  simulation  should 
alienate  the  team  members,  in  the  sense  of  repeatedly  injecting  forceful 
reminders  that  the  tactical  situation  is  synthetic.  The  literature  of  team 
training  is  inconclusive  insofar  as  insights  into  the  necessary  fidelity 
of  simulation  for  effective  training  are  concerned.  Perhaps  the  best  guideline 
for  the  system  designer  to  follow  is  a  rule  of  thumb  which  says  that,  within 
the  constraints  of  system  pragmatics  and  driven  by  system  requirements,  the 
simulation  subsystems  should  aspire  to  operational  transparency:  as  much 
as  possible,  the  output  of  each  simulation  should  be  indistinguishable  from 
its  real-life  counterpart. 


77 


N AVTR AEQUIPCEN  80-C-0095  -1 


Figure  11.  Components  of  an  automated  training  system. 
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The  simulations  which  are  required  by  the  team  training  demonstration 
system  are  enumerated  below.  For  each  simulation  subsystem,  a  top  level 
description  of  its  functional  requirements  is  given.  Broadly  speaking,  the 
simulation  subsystems  can  be  divided  into  two  general  groups:  team  member 
simulations,  and  all  the  remaining.  "All  the  remaining"  includes  all  the 
required  tactical  environment  simulations  such  as  physical  environment,  detection, 
weapons,  platform  motion,  and  supporting  personnel.  These  tactical  envir¬ 
onment  simulations  are  taken  up  first.  Then  the  simulations  of  team  members 
are  described.  The  general  approach  has  been  bo  make  the  tactical  environment 
simulations  good  enough  to  seem  reasonably  realistic,  but  as  simple  as  possible. 
The  team  member  simulations  will  be  considerably  more  complex. 

One  issue  of  terminology  should  be  dealt  with  here.  At  times,  a  given 
simulation  subsystem  is  referred  to  as  a  "model."  The  word  "model"  is  to 
be  understood  to  mean  a  representation  of  an  existing  object,  process,  or 
person  by  mathematioal  formulae  or  other  software.  We  do  not  intend  the 
word  to  mean  a  person  or  object  serving  as  an  example  to  be  imitated  or  oompared 
with.  This  distinction  must  be  made  because  the  team  member  models  will 
indeed  function,  in  some  sense,  as  exemplars,  and  oonfusion  of  these  two 
senses  of  the  word  "model"  must  be  avoided. 

THE  SYNTHETIC  TACTICAL  ENVIRONMENT 

Several  models  are  required  to  oreate  a  credible  simulated  tactical 
environment.  Those  elements  of  the  taotlcal  environment  which  must  be  modeled 
include  the  outside  physical  environment,  the  motion  of  surface,  subsurf aoe, 
and  air  platforms,  the  capabilities  of  the  various  ASW  weapons  systems,  the 
detection  subsystems  available  to  the  ASW  team,  and  enemy  maneuvers  and  tactics. 
Each  of  these  is  discussed  in  greater  detail  below. 

PHYSICAL  ENVIRONMENT  MODEL.  This  simulation  subsystem  will  essentially  be 
static  and  non-evolving.  Its  function  is  to  provide  a  limited  representation 
of  a  simulated  outside  world  for  the  benefit  of  the  other  simulation  models. 
The  principal  features  of  the  ASW  physical  environment  which  will  be  represented 
are  wind,  ocean  state,  ocean  thermal  profile,  visibility,  and  weather  state. 
Wind  and  ocean  conditions  are  the  most  important  among  these.  Wind  will 
be  maintained  as  a  number  on  the  Beaufort  scale,  and  sea  state  will  be  recorded 
as  well  as  the  presence  and  location  of  pertinent  ocean  thermal  layers 
All  the  physical  state  variables  shall  be  capable  of  being  set  by  the  instructor 
at  the  beginning  of  an  exercise  and  shall  remain  constant  throughout  a  given 
exercise. 

PLATFORM  MOTION  MODELS.  The  motion  model  simulation  subsystem  must  be  devoted 
just  to  the  job  of  moving  the  air,  surface  and  subsurface  tracks  which  are 
involved  in  the  problem  in  accordance  with  their  particular  dynamic  properties. 
This  subsystem  maintains  motion  model  minimum,  maximum,  and  nominal  values 
for  motion  parameters  such  as  speed  and  acceleration.  It  also  holds  turning 
parameters  for  surface  and  subsurface  tracks.  These  parameters  include  advance 
and  transfer.  The  motion  model  module  also  contains  the  dynamical  equations 
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required  to  update  the  position,  heading,  and  speed  of  all  the  tracks.  Sinoe 
the  geographic  scope  of  the  tactical  problem  area  is  relatively  small,  tb t 
tactical  arena  w<ll  be  modeled  as  a  plane,  and  all  positions  of  tracks  will 
be  reported  in  coordxurites  (x,  y,  2)  relative  to  this  piano.  The  x -  and 
y-  coordinates  measure  displacements  in  this  p.i  an*  relative  to  a  fixed  origin, 
and  the  2-  coordinate  represents  the  height  above  or  depth  below  this  plane. 

The  equations  of  dynamics  in  this  module  shall  permit  realistic  smooth 
changes  in  speed  and  heading  simultaneously,  and  shall  admit  cumulative  errors 
in  position  no  greater  than  50  yards  in  30  minutes  of  exercise  time. 

Motion  model  data  for  the  following  kinds  of  tracks  must  be  maintained 
by  this  module:  destroyers  (DD-963),  fixed  wing  aircraft  (P3),  LAMPS  heli¬ 
copters,  and  diesel-electric  submarines. 

WEAPONS  MODELS.  The  weapons  capabilities  of  the  ASW  attack  force  will  be 
simulated  by  the  weapons  control  module.  This  module  will  simulate  at  least 
torpedo  and  ASROC  type  weapons,  including  the  numbers  available,  the  dynamics, 
the  search  pattern  settings  (where  appropriate),  the  maximum  effeotive  ranges, 
and  the  dependence  on  ocean  conditions.  This  module  must  also  admit  a  probability 
model  for  determining  t^ie  consequences  of  a  given  weapons  launching  in  terms 
of  a  damage/destruction  evaluation  for  the  target  (and,  for  that  matter, 
for  any  friendly  tracks  in  the  vicinity) .  The  weapon  types  and  numbers  available 
to  the  ASW  team  should  be  amenable  to  instructor  control  at  problem  set-up 
time.  For  those  weapons  for  which  it  is  appropriate,  the  searoh  setting 
shall  be  capable  of  being  set  on  oommand  by  a  team  member.  This  module  shall 
be  designed  and  created  flexibly  enough  so  that  additional  weapon  capabilities 
oould  be  added  without  a  significant  re-design  effort. 

DETECTION  MODELS.  Several  detection  simulation  modules  are  required  to  model 
the  action  of  the  various  detect:'  •  capabilities  of  the  ASW  ship  and  supporting 
aircraft.  These  detection  capabilities  include  active,  passive,  and  dipped 
sonar,  as  well  as  magnetic  anomaly  detection,  radar,  and  visual  detection. 

£anqr .  The  most  highly  developed  of  the  detection  models  must  be  those  for 
sonar,  inasmuch  as  sonar  is  the  single  most  significant  tool  of  detection 
in  the  ASW  arena.  Sonar  shall  be  modeled  by  means  of  the  standard  sonar 
equations  for  active  and  passive  sonar  which  are  used  to  calibrate  operational 
sonar  systems.  Furthermore,  the  sonar  models  shall  take  into  account  noise 
from  the  baffles,  the  baffle  angle,  and  ocean  turbulence  (knuckles)  created 
by  turns.  The  object  of  this  additional  sophistication  is  to  simulate  realis¬ 
tically  the  diminution  or  loss  of  sonar  return  when  the  ship’s  maneuvers 
inhibit  effective  use  of  the  sonar.  The  output  of  the  sonar  system  shall 
be  a  number  indicating  strength  of  contact,  together  with  the  bearing  of 
that  contact  for  passive  sonar,  and  bearing  and  range  for  active  sonar. 
Doppler  shift  information  (up,  down,  no  Doppler)  shall  also  be  produced  by 
the  sonar  model. 
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Magnetic  Anomaly  Detection.  The  magnetic  anomaly  detection  (MAD)  simulation 
subsystem  shall  be  capable  of  modeling  the  action  of  an  airborne  MAD  device 
In  the  short-range,  low  altitude  searoh  for  looal  disturbances  in  the  earth's 
magnetic  field  produced  by  a  submarine  and  the  process  by  which  this  disturbance 
can  be  detected.  Environmental  variiblea,  especially  sea  state,  shall  be 
taken  into  account  by  this  model.  The  output  of  ihe  MAD  module  will  be  anomaly 
deteoted,  range  and  bearing,  or  no  detection. 

Radar.  The  radar  simulation  subsystem  shall  function  principally  to  model 
the  .  idar  looation  of  friendly  air  and  surfaoe  vessels  m  the  ASW  foroe. 
It  shall  have  the  additional  peripheral  function  of  modeling  radar  returns 
from  a  submarine  or  its  perisoope  should  *  ie  submarine  be  operating  at  or 
near  the  ocean  , urfa^e  and  within  radar  l.nge.  The  radar  model  shall  be 
capable  of  simulating  the  effect  cf  rain  and  other  weather  on  radar  ranges. 
The  output  of  this  module  shall  be  positions  for  all  tracks  (friendly  and 
hostile)  which  are  within  the  effective  radar  range. 

■YiaUftl.fifltmU.ttB*  Detection  tracks  by  ordinary  visual  sighting  shall 
also  be  simulated  to  the  degrc  .juired  to  produce  reports  of  visual  submarine 
sightings  by  air  or  surfaoe  tracks  when  a  submarine  would  realistically  be 
seen.  Also,  t’’,e  visual  detection  model  shall  be  called  upon  to  determine 
area-clear  information  for  t.he  bridge  model, 

ENEMY  MODEL.  The  eneoj  shall  be  rc  "esentiid  by  one  submarine,  and  the  enemy 
aimulatlon  subsystem  shall  b.  oap-'hle  of  expanding  to  tnoluda  at  least  two 
submarines.  The  primary  oo^oern  initially  should  be  to  oreate  a  model  of 
one  conventional  diaael  eleotrio  submarine  with  limited  intelligence.  This 
model  should  be  capable  of  pursuing  a  oourse  toward  a  specified  destination 
and  should  be  able,  if  the  instructor  chooses,  to  pursue  an  evasive  approaoh 
when  in  danger  of  attack.  It  follows  that  the  submarine  model  should  have 
i'he  capability  of  detecting  an  ASW  ship  using  passive  sonar.  The  enemy  model 
stall  be  deaigned  to  allow  expansion  cf  the  submarine's  intelligence  as  well 
us  to  permit  broadening  of  the  aubmarine's  capabilities  vis-a-vis  dynamics, 
weapons,  and  detection.  The  design  should  be  flexible  enough  to  allow  incor¬ 
poration  of  independent  work  on  the  r  deling  of  Intelligent  adversaries, 
ever,  though  the  intelligence  of  the  initial  enemy  model  will  be  rather  limited. 

MC  DELS  OF  ASW  SUPPORT  PERSONNEL 

Because  the  ASVJ  team  consisting  of  ASWOC,  ASWFCO,  and  ASAC  is  not  an 
isolated  entity  but  r ether  a  subteam  of  a  larger  group  engaged  in  ASW,  the 
simulated  oombat  environment  with  which  the  subteam  interacts  must  include 
adequate  models  of  support  personnel  to  simulate  a  full  ASW  group.  The  support 
p  rsonne]  who  are  relevant  include  ownship  personnel  (bridge,  plotters,  surfaoe 
watoh  ofiioer,  sonar  operator)  assist  ship  personnel,  supporting  aircraft 
personnel,  and  the  searoh  and  attook  unit  (SAU)  ooomandar.  "V.  support  personnel 
Simula  ten  subsystems  shall  model  those  aspects  of  the  behavior  of  ASW  group 
membsr ’  ,hiih  impinge  directly  on  the  ASWOC,  the  ASWFCO,  or  the  ASAC  in  the 
course  of  the  ASW  exercise.  In  particular,  communications  by  support  personnel 
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to  any  of  these  team  members  must  be  simulated,  both  with  respeot  to  form 
(actual  speech  to  be  generated,  or  button  action  to  be  simulated)  and  content 
(what  is  spoken,  what  information  is  transferred  by  button  action).  Furthermore, 
decisions  made  and  actions  taken  by  the  support  personnel  must  be  simulated 
insofar  as  they  influence  either  further  actions,  decisions,  or  communications 
by  the  team  members  or  the  general  course  of  the  ASW  problem. 

The  more  general  comments  in  an  earlier  section  about  simulation  in 
general  apply  here,  too.  The  output  of  these  support  personnel  simulation 
modules  is  the  most  significant  element;  the  means  by  which  the  output  is 
derived  is  less  important.  For  example,  the  model  of  bridge  personnel  need 
only  simulate  communications  made  by  bridge  personnel  to  team  members  and 
responses  to  team  member  communications  (in  terms  of  both  producing  simulated 
spoken  acknowledgments  and  causing  changes  in  the  ship’s  simulated  speed, 
heading,  etc).  A  detailed  model  of  bridge  operations  is  thus  not  appropriate. 

In  several  cases,  these  models  of  supporting  personnel  are  not  models 
of  Individual  persons,  but  instead  represent  collective  models  of  persons 
engaged  in  one  area  of  the  technical  arena.  A  single  model  of  all  the  assist 
ship  personnel  is  adequate,  for  example,  and  no  further  distinction  or  breakdown 
of  functions  is  necessary.  In  the  case  of  each  of  the  supporting  personnel 
models,  it  is  most  important  that  that  model  initiate  communications  at  appro¬ 
priate  times  which  are  correct  both  in  form  and  oontent,  and  that  the  model 
respond  appropriately  to  the  actions  and  communications  of  eaoh  of  the  team 
members.  In  the  sections  whioh  follow,  each  supporting  personnel  model  is 
taken  up  and  discussed  briefly. 

BRIDGE  MODEL.  The  simulation  of  ownship  bridge  personnel  shall  model  all 
appropriate  oommunioations  from  the  bridge  to  the  team  members,  whether  such 
communioation  is  a  response  to  ft  tew  timber  or  is  initiated  by  the  bridge. 
Furthermore,  the  bridge  model  will  respond  iu  a  realistic  fashion  to  ASWOC 
or  ASWFCO  requests  for  ohanges  ir.  f..e  .hip's  course  and  speed.  The  bridge 
model  is  also  responsible  for  'leratjwg  a.vi  a*  ee-olear  message  before  a  weapon 
is  fired. 

SONAR  OPERATOR.  There  ohall  bu  a  model  of  u  capable  sonar  operator  whioh 
simulates  all  relevant  communications  from  the  sonar  section  to  the  team 
members,  whether  such  communications  ar*.  responses  to  team  mmbei  oommunioations 
oi  are  initiated  by  sonar  perse  nel.  The  sonar  operator  model  shall  report 
the  output  of  the  physical  sonar  model  and  the  status  of‘  sonar  in  a  timely 
manner  to  the  ASWFCO. 

ASSIST  SHIP  PERSONNEL.  The  model  of  assist  ship  personnel  shall  simulate 
all  appropriate  oommunioations  from  the  assist  ship  to  any  cf  the  team  members, 
including  reports  of  sonar  contact  and  directions  for  ship  motion.  The  assist 
ship  model  shall  also  be  responsible  of  producing  oourse  and  speed  changes 
whioh  are  required.  This  simulation  subsystem  must  also  model  responses 
to  team  member  octanur-'.oations  and  oust  model  communications  which  are  ord<  T,‘*”Uy 
initiated  by  assist  ship  personnel. 
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AIRCRAFT  PERSONNEL.  Collective  models  of  aircraft  personnel  are  required 
for  the  following  aircraft:  fixed  wing  aircraft  (P3)  and  LAMPS  helicopters. 
Each  model  must  simulate  communications  with  team  members  which  are  realistic 
in  form  and  oontent.  In  particular,  magnetic  anomaly  detection  (MAD)  reports, 
sonar,  radar,  and  visual  oontaots  with  the  enemy  submarine  shall  be  simulated. 
Responses  to  directions  issued  by  the  ASAC  shall  be  made  both  by  appropriate 
simulated  aircraft  motion  and  by  simulated  speech. 

SEARCH  AND  ATTACK  UNIT  COMMANDER.  The  simulation  of  the  SAU  commander  shall 
model  the  SAU  commander's  supervisory  role:  this  simulation  subsystem  shall 
be  capable  of  responding  to  the  ASWOC's  communications  and  requests  for  permission 
to  take  various  actions.  This  model  shall  also  have  the  capability  to  initiate 
communications  so  as  to  be  able,  for  example,  to  countermand  decisions  of 
the  ASWOC.  The  SAU  oommander  model  shall  also  be  responsible  for  generating 
reports  to  the  ASWOC  regarding  ship  formation,  the  assignment  of  search  seotors, 
the  ohoioe  of  a  search  plan,  and  required  oourse  and  speed  changes. 

PLOTTERS.  Simulation  of  the  plotters  at  the  dead-reokoning  tracer  shall 
provide  the  capability  for  confirming  the  cone  of  oourses  and  limiting  lines 
of  approaoh  information. 

SURFACE  WATCH  OFFICER.  The  surface  watoh  officer  model  shall  have  the  capability 
to  determine  oourses  for  a  zig-zag  approaoh  and  shall  also  have  the  duty 
of  generating  an  area-clear  message  before  a  weapon  is  fired. 

MODELS  OF  THE  TEAM  MEMBERS 

Perhaps  the  principal  item  which  makes  the  proposed  team  training  demon¬ 
stration  system  unique,  aside  from  its  obvious  direction  toward  teams  instead 
of  individuals,  is  that  it  shall  provide  a  simulation  of  eaoh  team  member 
with  a  fidelity  sufficient  to  allow  the  oomputer  models  to  funotion  in  the 
absenoe  of  the  corresponding  team  members. 

This  notion  of  "simulating  missing  team  members"  is  not  taken  lightly. 
A  number  of  issues  -  ranging  from  cognitive  psychology  to  software  design 
-  are  involved  Just  in  developing  an  approaoh  to  the  problem.  How  shall 
the  full  range  of  knowledge,  experience,  and  intuition  of  a  human  team  member 
be  modeled?  Is  it  possible? 

A  CONCEPTION  OF  THE  PROBLEM.  Let  us  begin  with  an  admittedly  terribly  simplistic 
view  of  a  human  being.  A  human  being,  in  this  conception,  in  a  kind  of  "black 
box"  processor  which  aocepts  inputs  from  the  world  using  the  conventional 
human  sense  organs,  and  which  produces  outputs  in  both  verbal  and  haptic 
(button  pushing)  form.  Internal  to  this  black  box  processor  is  a  variety 
of  processing  activities,  some  of  which  we  oan  infer  (for  example,  by  intro¬ 
spection)  and  some  of  whioh  we  oan  mimic  without  neoessarily  understanding 
the  details.  We  know  that  this  blaok  box  is  equipped  with  a  memory  and  conse¬ 
quently  is  capable  of  basing  its  activities  on  past  experiences.  Beyond 
this,  we  realize  that  the  blaok  box  is  by  no  moans  a  simple  logic  engine 
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and  that  its  means  of  arriving  at  decisions  and  talcing  aotion  are  baaed  on 
deduction,  intuition,  expectation,  emotion,  and  muoh  else  that  ia  not  well 
understood. 

In  the  constrained  world  of  the  CIC  during  ASW  activities,  the  team 
members  -  ASWOC,  ASWFCO,  and  ASAC  -  must  possess  the  specialized  knowledge 
needed  tc  function  at  their  jobs  in  the  tactical  ASW  environment*  Beyond 
this  specialized  information,  human  beings  bring  to  these  three  tasks  certain 
other  kinds  of  more  general  knowledge  -  among  other  things:  common  sense, 
basic  information  about  how  the  world  works,  and  intuition  developed  through 
experienoe.  In  order  to  make  simulations  of  the  team  members  operational 
and  credible,  the  models  must  address  not  only  basic  knowledge  about  taotics 
and  team  coordination  but  must  also  inolude  basic  kinds  of  common  sense  and 
experimental  understanding.  An  ASWOC  modej.  whioh  "knows"  the  basio  taotioal 
rules  for  pursuing  a  submarine  but  does  not  know  that  a  DD-963  is  constrained 
to  move  on  the  surfaoe  of  the  ooean  would  not  be  worth  muoh. 

WHAT  THE  SIMULATION  SHALL  DO.  The  goal  of  the  team  Leober  simulation  system 
shall  be  to  model  the  taotioal  and  the  common  senae  knowledge  required  of 
the  ASWOC,  the  ASWFCO,  and  the  ASAC  in  such  a  manner  that  the  computer  models 
shall  be  able  to  funotion  in  place  of  a  human  team  member  who  id  ?,bsent, 
and  suoh  that  each  model's  performance  is  qualitatively  like  that  of  its' 
human  counterpart. 

A  general  outline  of  the  funotional  requirements  of  the  t»am  memtef 
models  is  provided  in  the  paragraphs  whioh  follow.  More  spweifio  details 
about  the  functions  of  the  ASlifOC  model  are  provided  in  Appendxx  1. 

The  ASWOC  Model.  The  simulation  of  the  AStyflC  shall  oodel  the  ASWOC  s  aotivitie* 
in  using  the  tools  at  h^s  disposal  to  pursue,  to  search  for,  and  to  Ipr.aiize 
a  submarine,  In  a  timely  manner,  the  model  shall  compute  and  uaiid  to  the, 
speeoh  generation  software  all  those  mrssagos  appropriate  to  the  our^ant 
taotioal  situation.  The  model  ahal.  also  initiate  appropriate  button  aotion 
communications.  Beyond  this,  fcy  virtue  of  the  speech  understanding  software, 
the  ASWOC  shall  Mspcnc  to  properly  framud  oommuni cations  from  hvtaan  tern 
members,  to  oemput  rr-genereted  communicate.. mu  from  ASW  Support  personnel 
simulation  software,  and  to  conaol  e-tuv-oonaole.  communications  from  human 
team  members  or  simulated  support  personnel.  When  the  taotioal  situation 
oalls  for  it,  the  ASWOC  model  shall  be  capable  o’'  staking  competent  oontext 
dependent  decisions  and  of  taking  appropriate  lotion  and  making  appropriate 
oomraands  to  implement  its  decisions. 

The  ASWFCO  Model.  The  ASWFCO  model  snail  be  capable  of  undrrstand.bg  and 
generating  the  seme  kinds  of  communications  is  the  ASWOC,,  although  the  total 
volume  of  ocmmunloatic:ia  involving  the  ASWFCO  is  considerably  smaller.  Usycpn 
these  communication  capabilities,  the  ASWFCO  model  shall  be  principally  res¬ 
ponsible  t<r  the  prepared  on  and  launching  of  weapo?is,  fer  ooordimuing  tbn 
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solution  of  the  fire  oontrol  problem,  and  for  the  determination  of  water 
entry  point  and  weapon  danger  areas.  The  ASWFCO  model  shall  also  be  responsible 
for  monitoring  simulated  sonar  reports. 

Provision  shall  also  be  made  in  the  ASVPCO  model  for  the  ASWFCO  model 
to  take  over  the  attack  from  the  ASWOC  or  ASWOC  model. 

The  as  AC  Model.  The  ASAC  model  shall  have  the  same  general  kind  of  ooomunioation 
capabilities  as  the  ASWOC  and  ASWFCO  models.  In  addition,  the  ASAC  model 
shall  have  the  capability  to  direct  ASW  aircraft  in  the  localization  and 
tracking  of  a  submarine,  as  well  as  ASW  air  weapons,  oontrol.  The  principal 
responsibility  of  the  ASAC  model  shall  be  to  communicate  the  ASWOC* s  oommands 
and  other  relevant  taotical  information  to  the  aircraft  and  to  relay  alroraft 
status  and  tactioal  data  to  the  ASWOC. 

AN  APPROACH  TO  THE  PROBLEM.  What  approach  should  be  taken  in  implementing 
these  team  member  simulations?  From  an  examination  of  small  pieoes  of  the 
whole  team  member  simulation  problem  what  has  emerged  has  been  a  considered 
judgment  that  a  so-oalled  "knowledge-based"  simulation  is  oallad  for  to  satisfy 
the  many  demands  we  have  placed  on  these  models.  A  reoommended  form  for 
this  knowledge-based  simulation  is  yet  to  be  determined  pending  a  most  careful 
scrutiny  of  the  knowledge  used  by  each  team  member  in  an  ASW  problem  like 
the  one  described  in  our  soenario.  Tentatively,  our  recommendation  would 
be  to  use  a  generalized  production  rule  system  such  as  is  described  in  Appendix  I. 

Characteristic  features  which  the  general  knowledge  representation  structure 
must  have  are: 

a.  strong  compatibility  with  the  kinds  of  knowledge  present  in  a  tactical 
environment  like  the  ASW  CIC, 

b.  capabilities  to  admit  meta-knowledge  for  oontrol,  so  models  can 
know  the  extent  of  their  knowledge,  and 

o.  potential  to  incorporate,  in  addition  to  basic  knowledge  relevant 
to  the  taotioal  situation,  special  kinds  of  instructional  and  perfor¬ 
mance  measurement  knowledge. 

A  preliminary  design  for  &  knowledge-baaed  ASWOC  model  is  included  as 
Appendix  I  of  this  report. 

EVENT  DETECTION  REQUIREMENTS 

An  event  detection  capability  is  required  to  enable  the  system  to  "know" 
what  is  happening  during  training.  This  capability  sh?.ll  be  distributed 
auong  a  number  of  procedures,  each  designed  to  monitor  and  interpret  data 
from  a  particular  information  ohannel.  These  individual  prooedurea  must 
format  information  and  send  it  to  the  centralized  message  routing  meahaniam 
for  distribution  to  the  appiioatious  procedures.  Thus,  for  example,  when 
the  speeoh  understanding  ospability  detects  that  the  ASWOC  has  spoken  and 
has  determined  the  meaning  of  the  communication,  it  must  formulate  a  message 
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consisting  of  a  unique  identification  number  vbj.th  specifies  that  the  message 
oontains  an  ASWOC  speeoh  communication.  the  channel  on  which  the  ooununi cation 
was  transmitted,  the  time  the  communication  oocnrred,  end  the  content  of 
the  communication.  It  must  then  forward  tor  mesaagu  to  the  oentralised  message 
routing  meohanism  which  will  in  tuirn  determine  the  set  of  procedures  and 
models  which  are  to  reoeive  the  message,  sand  it  to  each,  and  reoord  the 
message  for  subsequent  analysis. 

Events  can  be  classified  as  simple  cr  compound.  Simple  events  oorreapond 
to  single  events  in  the  simulated  world  such  as  the  utterance  of  a  particular 
verbal  communication  by  a  person  or  model,  or  ..the  sonar  detection  of  a  submarine. 
Compound  events  are  defined  through  Boolean  operations  upon  simple  events 
and/or  state  information.  An  example  of  a  compound  event  would  be  the  ship’s 
entering  the  torpedo  danger  area  (TDA). 

It  is  important  to  note  that  although  information  which  is  needed  to 
detect  trainee  errors  is  included  in  the  list  of  events,  trainee  erhora  as 
such  are  not  inoluded.  This  reflects  what  is  regarded  as  an  important  dasigu  . 
goal:  centralization  of  the  performance  measurement  capability.  All  decisions 
about  the  adequaoy  of  trainee  performance  ««Ut  be  the  work  cf  a  centralized 
procedure.  If  the  performance  measurement  rules  are  embedded  in  diverse 
event  detection  modules,  not  only  are  they  more  difficult  to  change  but  it 
is  more  difficult  to  tako  special  oircumatenoes  into  account  when  evaluating 
performance. 

PERFORMANCE  MEASUREMENT 

The  provision  of  automated  performance  measurement  means,  in  the  team 
training  aontext,  the  provision  of  the  capability  to  interpret  evicts  In 
the  training  environment  and  to  evaluate  trainee  and  team  responses  and  decis¬ 
ions,  The  data  generated  by  this  function  must  be  suitable  for  supporting: 

e  civ-lint  feedback  to  individuals 

e  on-line  feedbaok  to  the  team 

a  post-i  uir,  debriefing  of  individuals 

e  post-run  debriefing  of  the  team 

e  qualitative  performance  assessment  for  instructor  uec 
*  rooorvl  keeping 

e  acaptivw  problem  selection 

«  diagnosis,  prescription,  remediation 

e  statistical  analysis  and  norm  development 

While  not  all  of  these  features  will  be  provided  in  the  first  stage 
implementation  (o.g.,  diagnosis,  etc,)*  it  irs  nonetheless  important  to  recognize 
till  of  these  potential  uses  when  specifying  the  functional  requirements  of 
the  performance  measurement  capability. 
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A  variety  of  performance  measurement-related  funotions  will  be  needed 
in  order  to  provide  outputs  suitable  for  these  diverse  needs.  These  are 
disoussed  in  the  paragraphs  which  follow. 

ERROR  DETECTION.  There  is  a  class  of  errors  of  omission  and  commission  which 
oan  be  def lned,  a  priori,  as  being  wrong.  The  performance  measurement  system 
must  be  capehle  of  detailing  and  reporting  errors  of  this  class.  It  is  important 
that  this  capability  be  flexible  and  easily  enlarged  and  modified,  because 
it  it  inevitable  that  old  rules  will  ohange  and  new  rules  will  continue  to 
be  articulated. 

QUALITATIVE  PERFORMANCE  EVALUATION.  Error  reports,  in  themselves,  do  not 
provide  sufficient  information  for  performance  evaluation.  In  particular, 
the  instructor  needs  an  overall  assessment  of  the  quality  of  team  performance, 
and  the  system  itself  needs  a  measure  by  which  to  adaptively  scale  the  difficulty 
of  succeeding  problems.  In  early  development  work,  this  qualitative  evaluation 
oan  take  the  form  of  an  average  of  weighted  error  score*.  In  time,  as  performance 
data  <tre  oolleoted,  statistical  techniques  can  be  employed  tc  identify  faotors 
relevant  to  good  performance  and  the  contribution  of  the  various  measures 
to  the  estimate  of  those  faotors.  This  information  oan  ther.  be  incorporated 
in  the  automated  scoring  algorithm. 

Problem  Ar«,ft.a  in  Evaluation  of  Team  Performance.  The  ASW  team  training  oontext 
provides  special  challenges  to  the  designer  of  a  performance  measurement 
and  evaluation  system.  In  the  first  plaoo,  the  simple  enumeration  of  error 
events  for  the  three  individuals  and  the  team  becomes  exceedingly  oomplex 
in  an  emergent  situation,  Then  too,  the  opeolf ioation  of  toleranoes  for 
each  error  condition  is  difficult.  The  design  and  implementation  of  modules 
to  perform  the  evaluation  is  laborious  and  represents  at  least  a  duplicate 
implementation  of  the  performance  rules.  Just  to  obtain  consistency  between 
the  various  implementations  is  no  small  task.  An  example  may  clarify  the 
point.  Suppose  the  following  rule  is  extraoted  from  the  expert,  "Person 
P  shall  perform  notion  A  within  a  sale  period  of  time  from  event  E."  Suppose 
further  that  the  expert  la  pressed  until  he  defines,  Ka  safe  period  of  time" 
to  be  "within  10  seconds."  Using  a  traditional  approach,  this  rule  is  embedded 
in  the  model  of  person  P  in  such  a  way  that  the  model  lo  constructed  to  detect 
E  and  perform  A  in,  say,  five  seconds.  The  rule  is  ooded  again  in  the  performance 
measurement  logio  in  a  slightly  different  form.  Perhaps  when  E  occurs  It 
will,  schedule  an  omission  check  procedure  in  10  +  2  seconds  (giving  the  trainee 
a  fudge  faotor)  which  will  determine  whether  A  occurred  within  the  appropriate 
time  frame.  (Of  course,  there  are  many  other  ways  to  do  this  -  the  point 
is,  though,  that  the  requirements  of  the  performance  measurement  prooeon 
are  alight ly  different  from  those  of  the  model  team  member  prooeas.)  The 
;ule  must  be  ooded  a  third  time  to  provide  feodbaok  about  the  error  to  the 
trainee,  and  will  probably  take  the  form  of  the  English  aentenoe,  "You  failed 
to  take  action  A  within  10  seconds  of  event  E."  If  oooputer  aided  instruction 
fCAI)  is  provided,  the  rule  will  end  up  being  ooded  at  least  once  or  twice 
more  in  the  instructional  materials  as  well. 
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Suppose  all  of  this  has  been  done.  It  is  Inevitable  that  once  the  system 
is  working'  it  will  be  discovered  that  a  safe  period  of  time  does  not  equal 
10  seconds.  Mo  matter  how  "table  driven"  the  system  is,  the  difficulty  involved 
in  getting  the  ohange  implemented  should  be  obvious. 

Another  problem  whioh  will  arise  with  equal  inevitabllty  is  that  there 
will  be  times  when  the  trainee  is  unable  to  perform  the  required  action  because 
he  is  required  to  perform  some  other,  more  important,  aotion.  It  is  difficult 
to  prediot  all  of  the  possible  conflicts  of  this  sort  in  advance  and  to  take 
them  into  consideration  in  the  automated  performance  monitoring  capability. 

Finally,  the  method  has  the  profound  shortcoming  of  being  unsuitable 
for  generating  positive  feedbaok.  The  best  that  can  be  said  is  that  the 
trainee  made  no  errors. 

A  Different  Approach.  There  is  a  different  approach  to  performance  measurement 
which  may  enable  the  problems  described  above  to  be  surmounted  more  graoefully 
and  effectively  than  with  the  traditional  approach.  In  fact,  one  of  the 
primary  reasons  for  proposing  a  knowledge-based  approach  to  solve  the  team 
training  problem  was  precisely  to  accommodate  the  requirement  for  performance 
evaluation.  The  great  beauty  of  the  knowledge-based  approaoh  is  not  that 
knowledge-base  la  easier  to  debug  than  more  traditional  oode  (it  isn't), 
but  that  the  knowledge  itself  is  coded  only  once.  It  is  not  embedded  in 
oode  devoted  to  a  speoifio  purpose  and  is  not  divided  up  between  tables  and 
code.  Instead,  the  simulated  team  member  inference  engine  will  make  one 
use  of  the  production  rule  related  to  event  E,  while  the  performance  measurement 
inferenoe  engine  will  make  another  use  of  it.  To  provide  feedbaok,  a  third 
engine,  a  rule-to-English  translator,  oan  be  used  to  translate  the  rule  into 
English.  Other  ways  of  providing  feedbaok  are  also  possible  based  upon  the 
information  in  the  rule.  For  example,  Qoldstein  and  Crimson  (1977)  suggest 
associating  a  rationale  oonstruot  with  aaoh  rule  precisely  for  this  purpose. 

The  knowledge-based  approach  appears  then  to  offer  the  possibility  of 
providing  a  more  maintainable,  and,  in  some  ways,  a  more  efficient  way  of 
doing  traditional  event-related  performance  monitoring.  There  are  other 
potential  advantages  as  well.  A  crucial  element  in  the  design  of  a  knowledge 
base  which  Is  to  be  used  for  the  simulation  of  a  team  member's  behavior  is 
the  artioulation  of  the  rules  by  whioh  potential  actions  are  given  priorities. 
Some  tasks  oan  be  performed  in  parallel  while  others  oan, not,  and  the  simulated 
team  member  often  must  pick  the  best  of  several  good  alternative  oouroes 
of  action.  If  the  performance  evaluator  judges  the  trainee's  aotion  againut 
this  list  of  possibilities  with  assigned  priorities,  it  is  able  to  make  a 
qualitative  assessment  of  behavior  which  is  not  asoasaarlly  "wrong."  Furthermore, 
the  double  bind  alluded  t.o  in  the  paragraph  devoted  to  problem  areas,  wherein 
two  actions  defined  &  .priori  to  be  required  cannot  both  be  accomplished, 
is  completely  ciroumvented :  if  the  trainee  selects  the  better  alternative, 
the  system  is  able  to  recognize  that  the  situation  dictated  the  omission 
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of  the  other.  This  method  then  provides  the  potential  for  a  much  finer  resolution 
of  the  quality  of  performance  than  is  possible  with  simple,  event  driven, 
pass/fail  logics,  vhioh  oolleot  only  errors. 

The  foregoing  disoussion  has  fooused  upon  individual  behaviors  in  the 
team  oontext.  What  about  the  issue  of  team  performance  evaluation?  Does 
the  approach  offer  any  potential  benefit  there?  One  difficulty  has  been 
to  arrive  at  a  definition  of  effective  team  performance.  One  suggestion 
has  been  to  define  team  effectiveness  on  the  basis  of  the  degree  of  intelligence 
of  the  adversary  it  is  capable  of  overcoming.  This  is  obviously  a  good  measure 
for  use  by  a  fleet  commander  who  must  deoide  how  to  deploy  his  forces,  but 
it  may  not  be  of  muoh  pedagogical  value  other  than  to  discriminate  between 
those  teams  which  need  more  practice  and  those  which  do  not.  Harkening  back 
to  the  communications  control  foous  of  this  project,  it  is  dear  that  communi¬ 
cation  is  regarded  as  an  important  aspect  of  team  behavior  because  it  is 
important  that  eaoh  team  member  have  as  oomplete  and  aoourate  a  picture  of 
the  tactioal  environment  as  possible.  If  orucial  aspects  of  this  individual 
internal  representation  oould  bs  inferred  from  individual  performance,  and 
if  these  model  world  views  of  eaoh  team  member  oould  be  compared  with  one 
another  by  the  system,  it  might  be  possible  to  have  both  a  measure  of  t.e am 
functioning  and  a  basis  for  inferring  where  a  communication  breakdown  ooourred 
for  purposes  of  providing  helpful  feedback.  This  is  just  one  potential  inference 
technique  baaed  on  the  symbolic  representation  of  knowledge  which  might  ba 
pursued  to  the  benefit  of  team  training. 

Requirements.  The  potential  utility  of  a  knowledge-based  performance  measurement 
system  would  seem  to  Justify  serious  consideration  of  this  state-of-the-art 
approach.  There  will  be  significant  problems  to  be  overoooe  in  bringing 
these  ideas  into  a  practical  application.  It  is  also  the  case,  however, 
that  employing  a  more  traditional  approaoh  would  be  fravsht  with  difficulty 
and  the  result  would  likely  prove  to  be  of  limited  utility. 

The  first  step  in  actualizing  the  potential  knowledge-based  performance 
measurement  system,  suitable  bo  the  X8M  team  environment,  will  be  to  use 
the  same  knowledge  base  developed  for  the  team  member  simulations  to  Implement 
the  oheoks  for  error  events  Identified  to  date.  As  discussed  in  the  implemen¬ 
tation  plan  seotion,  the  first  stage  of  this  effort  shall  foous  upon  post-run 
performance  evaluation  based  upon  th®  record  of  events  oolleoted  during  a 
problem. 

RESEARCHER  CONTROL  IN  PERFORMANCE  MEASUREMENT  SELECTION.  The  researoher 
is  required  to  exeroiae  some  control  over  the  performance  measurement  system. 
It  oh»li  be  possible  for  the  receuroher  to  override  the  default  conditions 
and  speoify  whioh  team  member's  performance  is  to  be  evaluated,  and  to  specify 
that  a  set  of  rules  whioh  normally  would  apply  to  that  team  member's  behavior 
should  be  omitted  from  consideration, 

GRADING.  It  will  be  necessary  to  grade  eaoh  problem  iu  order  to  arrive  at 
soores  which  reflect  individual  and  team  performance.  The  knowledge-based 
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approach  offers  the  potential  for  arriving  at  a  qualitative  assessment  based 
upon  a  cumulative  comparison  of  student  decisions  to  the  decisions  made  by 
the  expert  model,  as  described  above.  This  is  properly  a  research  issue 
which  must  await  the  development  of  an  expert  model  team  member,  and  of  a 
definition  of  a  measure  of  olosaness  to  expert  behavior.  In  the  interim, 
however,  a  measure  of  performance  is  required.  This  grade  shall  he  computed 
based  upon  a  weighted  combination  of  errors  made. 

RECORD  KEEPING.  Complete  performance  records  shall  be  maintained  for  each 
problem.  These  records  shall  inolude  performance  data  specific  to  each  indi¬ 
vidual,  as  well  as  team  performance  data.  At  a  minimum,  the  reoords  shall 
oonsist  of  identifying  information,  problem  descriptive  information,  raw 
performance  data,  and  the  scores  assigned  by  the  training  system,  as  well 
as  textual  information  concerning  the  researcher's  assessment  of  the  problem. 
These  reoords  shall  be  accumulated  into  files  in  such  a  way  as  to  facilitate 
statistical  analysis  of  the  performance  data. 

PERFORMANCE  FEEDBACK  REQUIREMENTS 

Several  types  of  automated  performance  feedback  shall  be  provided. 
These  Inolude ; 

e  intrinsic  feedbaok  from  the  response  of  the  simulated  environment 
to  trainee  actions; 

*  optional  on-line  feedback  regarding  gross  errors  in  the  form  of 
SAU  commander  queries  and  recommendations; 
e  optional  on-line  prompting  when  an  error  is  deteoted; 
e  post-run  plot  of  the  exercise; 
e  optional  team  and  individual  performance  feedbaok; 
e  optional  display  of  errors  made  during  a  specified  portion  of  the 
problem;  and 

e  optional  performance  summary  reports  for  instructor  use. 

These  types  of  feedbaok  are  described  in  the  following  paragraphs, 

INTRINSIC  FEEDBACK.  The  simulated  operational  environment  will  provide  a 
aouroe  of  Intrinsic  feedbaok  to  the  trainee  in  the  sense  that  his  actions 
will  result  in  realistic  changes  in  that  environment.  Thus,  when  his  action 
is  well  chosen,  the  desirable  oonsequenoes  will  be  observed.  Likewise,  when 
his  aotlon  is  poorly  chosen,  he  will  have  to  deal  with  the  realistic  oonsequenoes, 

FEEDBACK  FROM  SAUC.  The  SAU  oommander  model  shall,  at  the  researoher's  dis¬ 
cretion,  provide  realistic  feedbaok  about  gross  errors  to  the  team  in  the 
form  of  queries  and  recommendations.  A  representative  sample  of  events  whioh 
should  oause  this  intervention  is  shown  in  Table  7,  along  with  the  dialogue 
whioh  the  SAUC  might  reasonably  initiate  in  each  case, 

PROMPTING  ON  ERROR  DETECTION.  A  research  issue  Involves  determining  the 
effioacy  of  immediate  prompting  with  the  oorreot  action  when  errors  are  deteoted. 
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TABLE  7.  SAMPLE  OF  ERRORS  RESULTING  IN  SAUC  INTERVENTION. 


Incorrect  classification  of 
suboarine 


ASWOC  fails  to  upgrade  olassifioation 
of  contact 


ASWOC  preps  a  bloodhound 


ASWOC  turns  in  a  direction  that  will 
take  him  out  of  his  sector 


SAUC  -  "Interrogative  classification 
on  your  oontaot  [track],  over.” 

-  Repeats  classification,  or  issues 
a  correction.  If  it  is  still 
inoorreot : 

SAUC  -  "What  are  the  criteria  for 
your  olassifioation,  over" 

•  Responds  with  oriteria 

SAUC  -  "Your  oriteria  meet  only 
the  requirements  for  [correct 
olassifioation],  over" 

SAUC  -  "Interrogative  classification 
of  your  oontaot,  over" 

-  Responds  with  upgraded  olassifioation 

SAUC  -  "Roger,  recommend  baoaatt, 
over" 

SAUC  -  Will  roger  the  turn  and  define 
new  seotore 
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Although  it  may  not  be  possible  to  provide  real-time  performance  measurement 
and  remediation  of  this  sort  in  the  Initial  stage  of  implementation!  the 
need  for  this  capability  must  be  recognized  and  the  feasibility  of  providing 
it  must  be  assessed. 

EXERCISE  PLOT.  As  a  post-run  debriefing  aid,  the  system  shall  provide  a 
graphios  display  presentation  and  optional  hard  oopy.  This  shall  include 
the  tracks  of  the  ASV  assets;  the  SAUC  ship,  training  ship,  and  helioopter. 
The  display  will  also  show  the  aotual  traok  of  the  submarine,  the  positions 
where  the  oontact  was  plotted,  and  the  weapon's  release  and  water  entry  points. 
Utilizing  different  oolors  for  the  actual  versus  plotted  submarine  traok, 
TDA  and  oone  of  courses  will  allow  these  data  to  be  easily  distinguished. 
This  display  will  serve  as  a  visual  aid  for  use  by  the  instructor  in  debriefing 
the  team.  The  hard  oopy  is  intended  primarily  for  researoher  use. 

The  display  lends  Itself  to  further  expansion  to  a  faster  than  real-time 
replay  of  the  proDlem,  with  computer  generated  annotation.  This  oould  serve 
as  an  excellent  souroe  of  feedback  for  the  trainees,  especially  if  the  annotation 
included  comments  about  good  decisions  and  aotions. 

Table  8  shows  the  characteristics  of  the  post-run  plot  and  describes 
the  additional  annotation  that  oould  be  provided. 

TEAM  AND  INDIVIDUAL  PERFORMANCE  FEEDBACK.  The  researoher  shall  have  the 
option  to  oause  the  system  to  convey  its  assessment  of  team  and/or  individual 
performance  to  eaoh  trainee.  This  will  allow  researoh  to  be  conducted  into 
the  effectiveness  of  the  type  and  timing  of  feedbaok  sinoe,  for  example, 
Hall  and  Rizzo  (1975)  have  provided  evidenoe  that  team  feedback  is  more  effective 
early  in  training  while  individual  feedbaok  is  more  effective  later.  Ultimately, 
the  system  oould  make  the  deoision  about  the  type(s)  of  feedback  to  provide 
baaed  upon  the  research  results.  A  oonoern  for  providing  positive  feedbaok 
must  pervade  the  design  so  that  the  feedbaok  provides  some  motivating  influence 
and  does  not  merely  serve  as  a  punisher. 

DISPLAY  OF  ERRORS.  It  shall  be  possible  for  the  researoher  or  trainee  to 
request  a  display  of  the  errors  which  the  system  detected  during  the  previous 
problem.  The  error  display  shall  be  either  a  display  of  all  errors,  or  a 
display  of  errors  made  between  speoified  times  during  the  exercise.  This 
report  shall  be  provided  at  each  trainee  station  if  requested,  and/or  on 
the  printer.  This  type  of  feedback  must  be  provided  sparingly  sinoe  it  tends 
to  be  overwhelmingly  negative,  oan  be  very  redundant,  and  may  not  distinguish 
between  errors  of  varying  severity. 

PERFORMANCE  SUMMARY  REPORTS.  The  system  shall  prepare,  at  researoher  request, 
performance  summary  reports  based  upon  information  in  the  student  files. 
Three  types  of  reports  shall  be  prepared;  1)  reports  oonoerning  all  teams 
trained  to  date;  2)  reports  concerning  the  progress  of  a  specific  team;  and 
3)  reports  oonoerning  individual  performance. 
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OTHER  POTENTIAL  TYPES  OF  FEEDBACK.  The  potential  benefits  to  training  of 
the  knowledge-based  approach  Go  not  stop  with  intelligent,  qualitative  perfom&noe 
evaluation.  A  natural  outgrowth  of  the  approach  is  to  provide  explanations 
of  performance  evaluations  and  to  suggest  better  strategies  under  the  given 
circumstances.  This  may  be  possible  if  an  expert  model  oan  be  developed 
which  employs  symbolio  decision  making  rules  akin  to  those  used  by  a  real 
expert,  as  opposed  to  using  a  purely  analytic  simulation  of  decision  making 
such  as  Rapoport's  (1967)  adaptive  case  model.  If  this  model  is  capable 
of  a  oertain  amount  of  baoktraoking  and  has  a  rationale  oonstruot  associated 
with  its  production  rules,  then  it  oould  potentially  provide  very  helpful 
explanations  and  instruction. 

ADAPTIVE  PROBLEM  SELECTION  REQUIREMENTS 

The  system  shall  have  the  capability  to  choose  subsequent  training  problems 
based  upon  its  measure  of  team  porf ormanoe.  The  researoher  shall  have  the 
capability  to  override  the  system's  selection. 

The  requirement  for  adaptive  problem  selection  in  this  research  devioe 
is  to  allow  the  capability  to  be  demonstrated.  For  the  demonstration,  the 
capability  will  be  limited  to  selecting  a  more  difficult,  or  less  difficult 
problem.  The  capability  for  more,  speoific  diagn  sis,  prescription  and  remediation 
is  not  required  for  several  reasons.  First,  with  the  subset  of  knowledge 
about  ASW  whioh  will  be  incorporated  into  the  system,  a  good  diagnosis  may 
not  be  possible.  Seoondly,  the  development  of  remedial  materials  is  a  research 
issue  outside  the  soope  of  the  present  development  effort.  Finally,  an  appro¬ 
priate  vehicle  for  much  remediation  is  some  form  of  individualized  CAI,  and 
that  capability  is  very  limited  in  the  system  as  presently  conceived. 

Table  9  lists  a  set  of  increasingly  difficult  variations  on  the  basio 
soenario.  These  variations  are  representative  of  the  type  of  problems  which 
oould  be  seleoted  by  the  adaptive  training  module  to  manipulate  problem  dif¬ 
ficulty,  although  not  all  will  be  implemented.  Problems  of  difficulty  equal 
to  the  ourrent  problem  will  be  generated  by  simply  manipulating  the  initial 
headings  of  the  players. 

COMPUTER  AIDED  INSTRUCTION  REQUIREMENTS 

Computer  aided  instruction,  understood  as  the  interactive,  presentation 
of  textual  materials  and  visual  displays  in  a  programmed  learning  context, 
is  an  extremely  valuable  training  strategy.  However,  it  doen  not  represent 
a  teohnologioal  risk  area  and  so  will  not  be  an  object  of  study  in  the  initial 
version  of  the  team  training  system.  The  CAI  approach  will  be  used,  however, 
in  the  oolleotion  of  representative  samples  of  voice  usta  and  for  system 
familiarization,  areas  crucial  to  the  success  of  a  speeoh  re  cognition- based 
training  system,  beoause  of  its  effectiveness  in  these  areas.  This  will 
provide  a  meaningful  demonstration  of  the  capability  in  the  team  training 
context  without  incurring  the  expense  of  the  development  of  ta«k  speoific 
CAI  courseware. 
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TABLE  9.  EXAMPLES  OF  PROBLEMS  OF  VARYING  DIFFICULTY  BASED  UPON  THE  SCENARIO. 

_ Ecafelaa _ Source  oL  Increased  PIfnsulix - 


Training  ship  will  be  given  sonar 
contact  well  outside  the  weapons1 
release  range. 


•  Training  ship  must  maneuver  the 
SAU  into  position  over  a  longer 
distance, 

e  ASWOC  must  make  more  decisions, 
communioate  more  with  the  assist 
ship,  and  communicate  more  with 
the  ASAC. 


t 


I, 


» 


Training  ship  will  be  given  two  ASW 
aircraft  to  control. 


Training  ship  will  be  given  a  second 
sonar  contact. 


For  any  of  the  above  problems, 
require  the  ASWFCO  to  take  control  for 
the  attaok. 


Training  ship  will  not  be  given  sonar 
contact  until  the  submarine  is  within 
range  where  an  urgent  attack  is 
required. 


In  any  of  the  above  problems,  cause 
the  submarine  to  change  course  after 
sonar  contact. 


In  any  of  the  above  problems,  cause 
the  submarine  to  fire  a  weapon  after 
the  training  ship  initiates  the 
attack. 


e  ASWOC  must  make  decisions  regarding 
aircraft  tactics. 

•  ASAC  must  control  two  aircraft 
through  these  tactics. 

•  ASWOC  must  evaluate  the  threat 
to  the  main  body  and  attack  the 
higher  threat. 

•  ASWOC  must  shift  oontrol  at  the 
proper  time. 

•  ASWFCO  must  maneuver  the  ship 
into  firing  position. 

•  ASWOC  must  make  a  snap  decision 
to  conduct  an  urgent  attaok  and 
maneuver  the  SAU  for  a  close-in 
attack. 

•  ASWFCO  must  prepare  and  fire  an 
urgent  attack. 

•  ASWFCO  must  override  the  system 
to  fire  the  weapon. 

w  ASWOC  must  adjust  to  counter  the 
threat, 

•  ASWFCO  must  recompute  the  fire 
control  problem. 

•  ASWOC  must  maneuver  to  avoid  the 
the  weapon,  then  to  regain  the 
firing  position. 

•  ASWFCO  must  recompute  the  firing 
solution. 


In  any  of  the  above  problems,  make 
the  threat  a  nuclear  submarine. 


•  Decreases  amount  of  playing  time, 
all  actions  must  be  performed 
more  quickly. 
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SECTION  VII 
IMPLEMENTATION  PLAN 

This  report  provides  a  functional  description  of  a  system  which  necessarily 
employs  new  hardware  technologies  and  for  wh^ch  a  very  new  kind  of  software 
design  is  being  recommended.  The  resul*  is  the  definition  of  a  system  built 
largely  on  unknowns,  intended  ultimately  to  solve  the  problem  of  automating 
team  training.  To  assure  success,  the  design  and  iLmplemehtation  must  proceed 
in  oarefully  considered  stages  in  which  only  a  few  unknowns  are  explored 
at  a  time.  The  fol  lowing  paragraphs  describe  a  thres-Stage  approach  to  the. 
development  and  the  rationale  behind  the  foous. of  each  stage. 

STAGE  1 

The  proposed  Stage  1  implementation  would  produce  a  system  in  whioh 
all  three  simulated  team  members  taokle  the  ASW  problem  against  a  submarine 
which  is  under  the  control  of  the  researcher.  The  simulated  team  members 
will  speak  and  perform  all  other  actions  that  will  be  expected  of  a  trainee. 
The  performance  monitoring  capability  will  grade  their  performance  after 
the  problem.  The  Stage  1  system  will  employ  a  minimum  hardware  configuration 
for  purposes  of  exploring  the  fundamental  adequaoy  of  the  knowledge- based 
approach  to  team  member  simulation,  Some  performance  monitoring  issues,  and 
speech  generation  techniques*  The  rationale  for  starting  with  these  research 
issues  is  described  in  the  paragraphs  whioh  follow. 

KNOWLEDGE-BASED  APPROACH.  It  has  been  argued  in  this  report  that  the  time 
has  ocme  to  attanpt  to  vastly  improve  automated  training  by  employing  a  knowledge- 
based  approach.  The  successful  oodification  of  team  member  knowledge  is 
important  for  several  reasons:  the  team  member  simulation  whioh  uses  this 
knowledge  will  allow  team  training  to  oocur  even  when  a  team  member  is  missing, 
a  common  problem  in  military  team  training;  the  knowledge  itself  will  provide 
the  basis  for  evaluating  team  member  behavior;  it  will  bo  the  source  of  prompts; 
and  it  will  provide  hypotheses  about  expected  speech  communications  which 
may  prove  crucial  for  guiding  speeoh  understanding. 

The  risk  in  the  approach  is  a  practical  one.  It  oust  be  determined 
whether  or  not  it  is  possible  tc  implement  the  approach  in  a  real-time  envi¬ 
ronment.  While  there  is  evidenoe  that  it  is  possible,  it  is  not  a  routine 
implementation  task  and  constitutes  a  risk  area. 

PERFORMANCE  MONITORING.  Performance  monitoring,  evaluation,  and  feedbaok 
in  the  team  training  environment  represent  virtual  unknowns.  What  is  well 
underwood  is  that  these  aspects  of  training  are  crucial  to  the  sucoess  of 
a  ti  ■  -siing  system.  Furthermore,  experience  has  shown  that  the  checking  for 
consistency  among  performance  measures,  the  oheckout  of  the  implementation, 
and  the  refinement  of  the  measures  as  experience  is  gained  on  the  system 
are  tasks  often  necessarily  relegated  to  the  final  stages  of  the  development, 
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with  the  undesirable  conaequenoe  that  one  of  the  most  crucial  training  j)  -«--uta 
of  the  bystdo  is  not  as  thoroughly  tested  as  it  should  bo  at  the  tl  .» 
first  trainee  begins  to  use  the  system* 

With  the  knowledge-based ; approach  to  performance  monitories,  there  at 
a  potential  for  early  and  thorough  Investigation  of  the,  performance ,  yjeitv^ *  v 
stage  1  attempts  to  capitalize  on  this  potential  in  order  to  give  the  auu  .  ited 
performance  monitoring  function  the  visibility  it  deserves  and  thh  develops  >nt 
time  it  needs.  A  further  important  aspect  of  the  choice  is  that  sxerolalng 
the  performance  monitoring  function  on  simulated  team  member  performance 
will  augment  the  debugging  of  the  model  team  member.  Finally  it  will  enable 
the  performance  monitor  to  be  debugged  before  the  oonfounding  influence  of 
potential  speeoh  misreoognitlon  is  introduced. 

This  approach  has  some  interesting  design  implications.  In  particular, 
the  simulations  and  the  performance  monitor  are  required  to  have  Beta  knowledge, 
that  is,  they  will  need  to  know  what  they  know.  In  this  w«y,  for  test  purposes, 
the  simulation  oan  be  given  incorrect  knowledge  wbioh  is  not  represented 
in  the  performance  monitor's  knowledge.  A  software  architecture  whioh  uses 
a  meta-knowledge  superstructure  is  consistent  also  with  a  parsimonious  approach 
to  knowledge  base  construction,  sinoe  a  great  deal  of  eaoh  team  member's 
knowledge  overlaps  that  of  other  team  members. 

SPEECH  GENERATION.  There  are  a  number  of  problems  in  the  speech  generation 
area  whioh  must  be  tadkled.  First,  team  members  get  verbal  communications 
from  a  large  number  of  people,  eaoh  of  whom  must  have  a  unique  voice.  Secondly , 
these  simulated  persons  talk  over  different  oirouits.  Finally,  these  commun¬ 
ications  must  be  retrieved  and  perhaps  constructed  to  provide  timely  Responses 
to  the  team  members.  ,  The  rationale  for  approaching  this  problem  in  the  first 
stage  ia  that  being  able  to  heb.r  the  interactions  will  greatly  faoilitate 
the  debugging  and  refinement  of  the  teas  member  models. 

SAMPLE  STAGE  1  WORK  PLAN.  Table  10  shows  an  outline  of  the  work  whioh  will 
need  to  be  performed  during  Stage  1,  and  Figure  12  shows  a  very  preliminary 
time  line  for  this  effort.  Tasks  1  and  2  are  primarily  intended  ■uo  validate, 
refine,  and  extend  the  specifications  In  the  functional  definition  as  necessary 
to  support  the  design  effort.  Part  of  task  1  and  task  3,  the  specification 
of  speech  understanding  requirements  and  the  implied  speech  recognition  hardware 
modifications,  are  shown  as  Stage  1  tasks  for  the  reason  that  our  experience 
with  procuring  similar  modifications  leads  us  to  reoommend  early  specification 
and  dialogue  with  the  manufacturer  to  ensure  the  availability  of  a  devioe 
with  the  requisite  capabilities  in  time  for  the  Stage  2  work  which  requires 
it. 

Task  4  involves  the  detailed  design  of  Stage  1  software  components. 
This  work  oan  proceed  to  some  extent  during  the  hardware  procurement  cycle. 
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TABLE  10.  OUTLJIE  Oh  STAGE  1  TASKS. 

l&ak _ _ . _ Rfldflrlpfcloa _ 

..  .  •  •  ,  .‘  ■  ■  , 

1  ,  Verify  and  enlarge  upon  the  specification  of  the  functional 

requirement*  f or t 

*  the  detailed  knowledge  about  .the  problem  , 

>  teas  member  knowledge  and  behavior. 

i':\  ';•>■  environment  ape cifi option  '> "'  //;<',  '  vv 

*  the  performance  measurement  system 

*  the  adaptive  logic  V 

"•  the  speech  generation  System 

*  the  resear oher/ instructor  interface 

*  the  speech  understanding  system 

2  Validate  proposed  hardware,  software  languages,  and  operating 
system  selections  for  the  Stage  1  system  and  Initiate  procurement 

3  Specify  necessary  modifications  to  existing  apeeoh  recognition 
hardware,  initiate  dialogue  with  manufacturer^) 

4  Design  software; 

*  the  knowledge  base 

*  the  executive 

*  the  team  member  models 

*  the  supporting  simulations 

*  the  performance  measurement  system 

*  the  adaptive  logic 

*  the  speech  generation  system 

*  the  researcher/ instructor  lnterfaoe 

*  tho  support  modules  (graphics  capability,  researcher/ 
submarine  interface,  etc*) 


.5 

6 


Y 


8 


9 


Design  any  special  purpose  hardware 
Implement  hardware  designs,  configure  hardware  system 
Implement  stand  alone  checkout  of  software 
Integrate  software,  system  level  checkout 
Fine  tunfc  system 


10  Evaluate  the  feasibility  of  the  approach  and  the  implications 

of  lessons  learned  for  subsequent  stages  of  tbe  development 


Ml ■'VMMXflrva*  V  ■—  — f  — -i.n-  we 
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Tasks  5  and  6  involve  the  fabrication  of  any  special  purpose  devices 
end  interfaces  that  may  be  required,  as  well  as  configuring  the  hardware 
equipment  suite. 

Tasks  T  and  8  relate  to  the  software  impl mentation  effort  and  the  inte¬ 
gration  of  ooftwars  system  components. 

Task  9  is  shown  in  recognition  of  the  fact  that  model  behavior  and  perfor¬ 
mance  measurement  system  operation  are  best  refined  in  the  oontext  of  a  working 
system.  The  researcher  needs  ample  time  to  hone  these  all  important  training 
sy^ttan  elements'  before  the  system  is  subjected  to  a  real  trainee. 

finally,  task  10  provides  the  opportunity  to  specify  guidelines  for 
the  next  stage  of  the  implementation  in  the  light  of  the  lessons  learned 
to  date. 

STAGE  2 

When  the  Stage  1  system  reaches  the  state  of  refinement  in  which  the 
adequacy  of  the  methods  and  hardware  has  been  demonstrated,  it  will  be  .soessary 
to  generate  a  detailed  Stage  2  plan  in  the  light  of  the  lessons  learned  in 
Stage  1.  The  foous  of  the  Stage  2  can  be  sketched  here,  hoover.  The  intent 
is  to  add  the  capability  to  support  the  training  of  one  team  member.  This 
will  entail  the  addition  of  hardware  and  software  to  the  Stage  1  system.  The 
hardware  will  include  a  speech  recognition  device,  probably  one  especially 
configured  based  upon  the  reoommendations  developed  in  Stage  1;  a  color  graphics 
capability;  and  additional  computer  support.  The  software  will  include  the 
speech  understanding  software  which  interprets  the  potentially  noisy  speeoh 
recognition  devioe  output;  the  interfaoe  between  speech  understanding  and 
the  rest  of  the  system;  speech  data  oolleotion;  the  full  computer  generated 
submarine  model;  additional  graphics  support;  and  perhaps  real-time  performance 
measurement. 

The  Stage  2  system  will  be  used  to  investigate  solutions  to  the  understanding 
of  connected  speech  in  the  ASW  CIC  environment  and  to  answer  questions  about 
the  effeotlve  use  of  feedback  and  the  use  of  performance  data  in  adaptive 
problem  selection.  It  will  also  provide  particularly  valuable  information 
about  the  effectiveness  of  the  simulated  team  members  in  functioning  with 
a  human  team  member. 

Stage  2  is  regarded  as  a  crucial  interim  stage  for  several  reasons. 
First,  even  though  a  body  of  knowledge  has  been  accumulated  about  understanding 
the  speech  of  one  person  talking  to  a  computer,  the  ASW  vocabulary  is  extensive, 
ani  some  provision  must  be  made,  if  possible,  for  understanding  language 
whioh  is  not  necessarily  perfectly  structured  according  to  rules  imposed 
for  the  sake  of  computer  recognition. 

Secondly,  the  efficacy  of  the  speeoh  recognition  hardware  modifications 
should  be  evaluated  before  further  device  procurement  is  i.._tiated 
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Thirdly,  although  Stage  1  will  insure  that  the  simulated  teas  members 
oan  funotion  as  a  team,  there  will  undoubtedly  be  much  to  learn  when  a  human 
funoticnn  as  one  of  the  team  members. 

Finally,  the  Stage  2  devioe  will  provide  the  researoher  with  a  tool 
for  studying  the  effeotive  uses  of  performance  data  for  feedback  and  adaptive 
problem  selection. 

STAGE  3  ' 

Stage  3  will  involve  extending  the  system  capacity  to  accommodate  all 
tbre*  team  members.  This  extension  Hill  be  based  upon  all  the  lessons  learned 
,in  Stages  1  and  2,  and  especially  upon  the  results  of  the  speech  understanding 
work.  The  new  speech  understanding  problems  which  will  arise  in  thi !. ‘-v  stage 
are  those  associated  with  understanding  the  communications  between  human 
team  members,  since  the  team  may  tend  to  revert  to  less  structured,  less 
intelligible  speech  between  themselves  than  is  used  in  oommunioating  direotly 
with  the  simulations.  It  is  neoessary  for  the  system  to  understand  this 
oonmunioaticn  as  best  it  can  in  order  to  be  able  to  detect  communications 
breakdowns.  This  will  likely  call  for  increasing  sophistication  and  refinement 
of  the  speech  understanding  logic,  and  support  frem  the  team  member  models. 

It  is  in  this  stage  also  that  research  into  issues  regarding  team  behavior 
oan  be  conducted,  especially  into  motivating  such  behavior  and  the  effeotive 
user  of  team  versus  individual  feedback. 
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SECTION  VIII 

CONCLOSIONS  AND  RECOMMENDATIONS 

This  document  has  presented  a  preliminary  system  design  for  a  team  training 
demonstration  system  based  on  extensive  analysis  of  the  team  training  problem 
in  an  ASW  environment.  It  is  believed  that  the  system  which  is  described 
here  would  provide  a  valuable  tool  for  further  detailed  investigation  of 
team  training,  and  would  contribute  significantly  to  the  development  of  actual 
automated  team  training  devioes.  It  is  recommended  that  the  staged  implementation 
plan  describe.-  in  this  report  be  carried  out. 

There  are,  however,  three  areas  in  which  short-term  action  is  recommended 
to  enhance  the  speedy  development  of  team  training  technology  and  to  improve 
existing  training.  These  will  be  discussed  in  the  following  paragraphs. 
The  reader  is  urged  to  review  the  chapter  which  presents  the  staged  implementation 
plan  inasmuch  as  two  of  the  items  which  follow  simply  highlight  features 
of  that  general  plan. 

SPEECH  TECHNOLOGY 

The  amount  of  communications  which  take  plaoe  in  a  Combat  Information 
Center  is  simply  staggering.  When  the  sheer  bulk  of  these  communications 
is  considered,  and  the  crucial  dependence  of  mission  success  on  communication 
is  noted,  it  becomes  clear  that  speech  technology  has  vital  importance  in 
the  development  of  an  automated  training  device  for  this  environment  and 
many  like  it.  Automated  performance  measurement— sorely  needed  by  overburdened 
instructors--is  not  practical  without  a  reasonably  reliable  automated  speech 
understanding  capability.  It  has  been  noted  that  understanding  cf  team  training 
issues  has  long  been  hampered  for  laok  of  detailed,  objective  performance 
measurement,  and  manual  methods  of  accomplishing  performance  measurement 
are  neither  praotioal  nor  adequate.  In  the  last  few  years,  automated  systems 
for  training  individual  students  have  been  built  whieh  successfully  integrate 
adaptive  training,  automated  performance  measurement,  and  automated  speech 
recognition  and  speech  generation  (Grady,  in  press;  Hioklin  et  al.,  1980). 

It  now  appears  that  the  time  has  arrived  when  these  emerging  technologies 
can  be  applied  to  team  training.  But  speech  technology,  particularly  speech 
recognition,  is  the  crucia1  element  in  this  scheme,  and  it  is  still  developing. 
It  is  strongly  recommended  that  the  state  of  speech  technology  be  regularly 
monitored  and  that  new  techniques  and  new  devices  be  evaluated  quickly  as 
they  appear. 

KNOWLEDGE-BASED  SIMULATION 

A  significant  consequence  of  this  study  and  design  effort  has  been  the 
recognition  that  a  knowledge-based  approach  toward  simulating  team  members 
is  very  desirable  given  the  complexity  of  the  team  training  situation  and 
the  complication  of  the  ASW  world.  The  advantages  of  having  a  knowledge-based 
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model  of  each  team  member  include  not  only  the  capability  of  providing  a 
high  fidelity  simulation  of  missing  team  members,  but  also  the  facility  for 
oomparing  expert  model  performance  to  that  of  the  trainee.  The  knowledge-based 
modeling  approach  thus  strengthens  the  capabilities  of  performance  measurement 
and  allows  for  more  intelligent  feedback  to  the  trainee.  Furthermore,  if 
the  knowledge  base  inoludes  rules  about  performance  measurement  and  training 
techniques,  the  knowledge-based  system  approaoh  beoomes  an  attractive  means 
for  automated  performance  measurement  and  adaptive  logic.  It  is  strongly 
reoommended  that  the  technology  of  knowledge-based  modeling  of  expert  behavior 
be  vigorously  pursued. 

COMMUNICATIONS 

The  current  study  hrs  recommended  a  training  approaoh  designed  to  promote 
strong  team  formation.  This  approach  includes  training  to  (1)  emphasize 
the  critical  communications  required  of  each  team  member,  including  both 
those  communications  properly  theirs  as  well  as  communications  made  to  backup 
fellow  team  members,  (2)  Instill  a  shared  sense  of  identity,  and  (3)  develop 
a  shared  set  of  goals.  Weaknesses  in  ASW  teams  which  have  been  observed 
during  training  include  inadequate  flow  of  information  between  team  members, 
misunderstanding  of  mutual  relationships,  and  vagueness  about  team  goals. 
While  current  training  formally  reoognizes  the  importance  of  team  oonwuni cations 
(see  Appendix  L) ,  there  is  little  or  no  explicit  emphasis  on  communications. 
It  is  reoommended  that  (1)  instructors  be  made  aware  of  the  need  for  training 
a  team  to  communioate,  as  well  as  being  informed  of  methods  to  develop  good 
team  communications  skills,  and  (2)  instructional  materials  dealing  with 
communication  issues  be  developed  for  early  use  in  the  team  training  syllabi. 
Because  team  communication  problems  are  so  crucial,  there  is  a  pressing  need 
for  even  partial  solutions  as  soon  as  possible. 
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APPENDIX  A 

TRAINING  TECHNOLOGY  BACKGROUND 

The  Naval  Training  Equipment  Center  ( NAVTRAEQOIPCEN)  hae  established 
the  efficacy  of  various  training  automation  teohnologiea,  notably  adaptive 
training,  automated  performance  measurement,  and  computer-based  speech  recognition 
and  voioe  generation.  The  combined  applications  of  these  technologies  have 
reduced  the  instruotor-trainee  ratio,  reduced  training  time,  and  increased 
training  standardization,  while  maintaining  high  quality  and  efficiency. 
These  observations  have  been  derived  from  flight  training  systems  and  ground 
controller  training  systems  (Hioklin  et  al. ,  1980;  McCauley  and  Semple,  1 9 60 ; 
Grady,  in  press).  This  project,  Team  Training  Through  Communication  Control, 
seeks  to  extend  the  benefit  of  these  technologies  to  the  team  training  envi¬ 
ronment, 

WHERE  DOES  THIS  PROJECT  FIT  INTO  TECHNOLOGY  DEVELOPMENT? 

A  complete  Team  Training  Automated  Training  System  will  encompass  the 
following  major  subsystems: 

•  Simulation  -  the  environment  for  prnetioe 

e  Replay  -  for  self-review  and  self-critique 

•  Performance  Measurement  -  the  basis  for  student  assessment 

•  Feedback  -  for  automated  evaluation  and  self-critique 

•  Adaptive  Logic  -  for  automated  Self-pacing 

•  Computer  Aided  Instruction  -  for  teaching  the  material 

•  Diagnosis  and  Remediation  -  for  "instruotorless"  training. 

A  review  of  the  history  of  some  familiar  Navy  training  programs  oharts 
the  development  of  these  various  subsystems. 

TACDEW  -  simulation,  replay,  very  simple  performance  measurement 

AFTS  -  simulation.  replay,  performance  measurement,  simple  feedback, 

simple  adaptive  logic 

GCA-CTS  -  simulation,  replay,  performance  measurement,  feedback,  adaptive 
logic,  simple  computer  aided  instruction,  simple  diagnosis  and  remediation 
ACE  -  ail  of  the  above,  none  of  it  simple! 

As  we  embark  on  the  team  training  program,  we  pose  the  question  of  what, 
if  anything,  makes  this  application  of  automated  training  system  design  concepts 
so  different?  Is  this  project  ooneerned  with  the  oontinued  development  of 
the  subsystems  named  above?  What  makes  this  project  truly  developmental 
in  nature,  and  not  just  a  re-application  of  the  technologies  used  in  those 
earlier  programs? 

A  considerable  level  of  research  is  being  conducted  in  team  training 
issues.  See,  for  example,  the  proceedings  of  the  Team  Performance  Workshop 
held  at  RAND  in  August,  1980.  This  wide  ranging  research  is  a  very  important 
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element  whose  impact  on  the  specification  of  an  eventual  operational  traine" 
will  be  significant.  NAVTRAEQUIPCEN' s  participation  in  this  research  is 
characterized  by  focusing  on  team  communications  and  training  automation 
as  important  aspects  of  tdsm  performance  and  team  training;  and  insisting, 
from  the  start,  that  the  results  of  the  RAD  have  ’’credibility  and  utility 
in  the  eyes  of  the  operational  Navy,  as  well  as  methodological  precisicn" 
(Rizzo,  1980).  Note  that  this  statement  does  not  neoessarily  imply  that 
the  RAD  tools  will  replace  or  even  augment  any  existing  training  program 
or  devioe.  It  does,  however,  demand  that  the  RAD  be  structured  around  real 
Navy  problems  and  certainly  not  be  oounter-productive  or  even  non-productive 
for  potential  operational  Naval  subjects.  It  demands  a  system  that  trains, 
but  not  neoessarily  a  Training  System. 

PURPOSE 

The  foous  upon  team  communications  and  trav  og  Automation  provides 
the  guideline  for  scoping  the  problem  in  tar-ms  of  .  ID  objeotivea,  goals, 
and  issues.  From  this  baseline  of  v'lear  and  concise  objectives,  the  need 
to  offer  credibility  and  utility  in  the  eyes  of  the  operational  Navy  provides 
the  framework  for  defining  the  operation*.1  and  functional  aspects  of  a  Researnh/ 
Demonstration  System.  This  name  captures  the  notion  that  it  is  a  demonstration 
to  the  operational  Navy  of  oertain  instructional  strategies  and  automation 
technologies  as  applied  to  a  training  problem  with  which  they  can  relate; 
ar.d  it  is  a  demonstration  to  the  RAD  community  of  these  same  strategies  and 
technologies  applied  in  a  experimental  tool  for  their  use  and  study.  It 
also  underscores  the  distinction  between  the  multi- fade ted  goals  of  this 
experimental  system  with  the  more  pragmatic  compromises  that  ere  required 
in  any  operational  trainer.  Most  importantly,  the  name  conveys  the  important 
role  of  this  system  as  a  research  tool  for  examining  the  crucial  questions 
which  must  be  answered  before  designing  an  operational  training  system. 

RAD  OBJECTIVES.  The  basic  RAD  question  posed  by  this  program  is:  Can  team 
performance  be  improved  through  automated  training  in  team  communications? 

More  specifically: 

What  is  the  nature  of  ,tq?m  communications? 

How  nan  team  communications  be  taught? 

How  can  it  be  practiced,  measured,  and  evaluated? 

How  oan  communication  problems  be  diagnosed  and  weaknesses  remediated? 

Reflecting  upon  these  questions  naturally  leads  to  other  questions, 
answers  to  vhioh  must  be  provided  before  an  efficient  and  effective  training 
devioe  oan  be  specified. 

Do  communication  skills  correlate  well  with  team  performance? 

If  so,  oan  team  communication  be  measured,  evaluated,  and  inter¬ 
preted  to  predict  team  performance?  Bow  does  this  differ  from 
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predictions  based  on  observation  of  trainee’s  communications  skills 
when  practiced  individually?  Can  a  standard  be  defined?  Can  we 
determine  the  communications  skills  of  a  "good"  team? 

Can  communications  be  controlled  sufficiently  to  adapt  the 
training  situation  to  the  unique  strengths  and  weaknesses  of  the 
individual  team  members  and  the  team  as  a  whole?  Is  communications 
a  good  adaptive  variable?  What  aspect  would  ona  want  to  a&ayv: 
informational  content'  frequency,  style,  etc.?  How  do  we  model 
this  range  of  communications  skills? 

Can  a  simulation  environment  which  stresses  the  oommunicati in 
channels  be  created  with  sufficient  fidelity  to  provide  valid  answers 
to  research  issues  addressed  within  that  environment?  Can  we  simulate 
verbal  communications  of  the  various  team  members  sufficiently 
realistically  to  provide  face- validity  to  the  trainee?  Can  the 
supporting  technologies  provide  what  to  say  (as  well  as  how  to 
say  it)  in  a  dynamically  changing  environment? 

What  are  appropriate  instructional  strategies  and  media  to 
be  used  in  teaching  team  communications?  What  is  the  role  of  CAI, 
video  disc-based  instruction,  the  simulator,  live  pradtice,  etc.? 

What  are  the  most  appropriate  feedback  techniques?  What  should 
be  the  feedbaok  content?  Should  feedbaok  be  direoted  to  the  entire 
team  or  just  the  individual  to  whom  it  applies?  When  must  the 
feedbaok  be  givens  immediately  or  after  a  mission? 


In  terms  of  R&D  objectives,  then,  the  purpose  of  this  program  is  to 
provide  an  experimental  tool  which  is  sufficiently  robust  and  flexible  to 
support  research  to  answer  these  and  similar  questions. 

SYSTEM  DESIGN  OBJECTIVES.  In  terms  of  system  design  considerations,  there 
are  in  fact  a  number  of  fundamental  differences  in  this  program.  One  difference 
includes  but  extends  beyond  the  fact  that  a  team  is  the  "trainee"  and  that 
communications  is  the  focal  issue.  A  particularly  Important  and  basic  difference 
which  will  significantly  effect  the  design  of  the  subsystems  named  above, 
is  that  we  are  attempting  to  apply  automated  and  adaptive  techniques  to  a 
primarily  emergent  situation  as  distinguished  from  a  primarily  established 
situation.  Because  of  its  significance,  we  will  examine  this  aspect  of  the 
program  throughout  various  sections  of  this  report.  Briefly  though,  these 
terms  are  explained  as  follows: 
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An  established  situation  is  one  in  which  (1)  all  action-relevant 
environmental  conditions  are  specifiable  and  predictable,  (2)  all 
action-relevant  states  of  the  system  are  specifiable  and  predictable, 
and  (3)  available  research  technology  or  records  are  adequate  to 
provide  statements  about  the  probable  consequences  of  alternative 
actions.  An  emergent  situation  is  one  in  whioh  (1)  all  action-relevant 


111 


■,  <a.:P 


NAVTRAEQUIPCEN  80-C-0095-1 


environmental  oonditions  have  not  been  specified,  (2)  the  state 
of  the  system  does  not  correspond  to  relied-upon  predictions,  and 
(3)  analytic  solutions  are  not  available,  given  the  ourrent  state 
of  analytic  technology  (Eoguslaw  and  Porter,  1962). 

It  is  important  to  note  here  that  the  authors  underscore  that  any  given 
context  is  aotually  a  oontinuum  of  situations,  the  end  points  of  which  are 
described  as  established  or  emergent. 

AFTJL  and  GCA-CTS  were  clearly  applications  biased  toward  established 
situations.  Flight  procedures,  flying  final  PAR  approaches,  conducting  GCA 
control,  are  all  tasks  vhioh  are  well  defined,  specifiable,  and  predictable. 
The  introduction  of  randomness  into  these  systems,  through  simulated  emergencies 
or  non-deterministic  pilot  and  wind  models,  in  fact  did  provide  some  unpredicta¬ 
bility  from  the  trainee's  perspective,  but  oertalnly  not  the  system's.  The 
response  by  the  learner  was  equally  well  specified  to  the  system's  simulation 
and  performance  measurement  subsystems  given  these  modifications  from  the 
norm.  To  the  extent  that  the  training  task  and  environment  beoome  less  well 
defined,  for  example  in  some  AIC  tasks,  the  training  system  designer  has 
resorted  to  a  rigidly  specified  soenario  which  in  fact  onoe  again  constrains 
the  situation  to  cause  and  effeot  relationships.  In  summary,  many,  if  not 
all,  applications  of  automated  and  adaptive  training  technologies  and  in¬ 
structional  strategies  have  been  applied  to  situations  which  are  more  established 
than  emergent. 

Now  if  one  were  solely  Interested  in  developing  "Just  another"  ASW  Trainer, 
albeit  with  some  automated  and  adaptive  features,  one  might  be  tempted  to 
adopt  an  approaoh  whloh  similarly  constrained  the  ASW  team  tasks  via  carefully 
defined  scenarios  and  relatively  "hard  coded”  (deterministic)  subsystems. 
Even  if  this  approach  proved  viable  (whloh  we  doubt),  the  desire  to  examine 
generic  team  tasks  and  to  extend  the  results  of  the  RAD  effort  to  other  team 
environments,  suggests  that  "Just  another"  trainer  designed  along  the  more 
traditional  guidelines  is  not  an  appropriate  objeotive  of  this  program. 

Muoh  of  the  basic  research  conducted  in  team  training  confirms  one'e 
intuition  that  team  training  devices  and  techniques  require  an  orientation 
toward  emergent,  unstructured  situations.  For  example,  Wagner  (1977)  reports 
that: 

An  emergent  situation  permits  a  more  realistic,  less  abstract 
approach  to  the  investigation  of  team  training  variables  than  does 
an  established  situation.  When  a  team  was  formed  in  the  laboratory, 
its  prior  motivational  characteristics  were  Important  determinants 
of  performance.  But,  when  teams  were  studied  while  carrying  out 
face-valid  activities,  prior  motivational  states  appeared  to  be 
less  Important.  What  was  important  in  this  c^se  was  training  in 
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coordination  such  that  team  members  became  fully  aware  of  their 
responsibilities  to  compensate  for  the  inabilities  of  others,  or 
to  overcome  temporary  problems  when  the  situation  oalled  for  it.1 

SUMMARY.  To  return  to  our  earlier  questions  then,  this  project  provides 
both  the  opportunity  and  the  requirement  to  extend  automated  and  adaptive 
training  into  situations,  like  A SW,  whioh  involve  team  activities  in  primarily 
emergent  situations.  We  are  challenged  to  push  the  state  of  analytic  technology 
to  provide  solutions  whioh  heretofore  be*e  not  been  available.  To  summarize 
then,  this  project  must  develop,  teat,  and  demonstrate  automated  training 
technologies 

•  as  applied  to  team  performance 

•  focusing  on  oommuni cations  control 

•  in  a  dynamic:  (A3W)  environment. 

ASSUMPTIONS  AND  CONSTRAINTS 

What  does  this  imply  about  the  system  configuration?  We  currently  envision 
a  demonstration  system  emphasizing  simulation  and  performance  measurement. 
The  Simulation  Subsystem  will  inolude  relatively  traditional  modules  for 
platform  motion  models  (aircraft,  surface  ships,  and  submarines),  detection 
models  (active  and  passive  sonar,  anJ  magnetic  anomaly  detection),  and  support 
personnel  (sonar  operators,  fire  control  specialists,  bridge  personnel,  pilots, 
etc.).  The  major  emphasis,  however,  will  be  placed  on  developing  new  teohniques 
to  model  the  principal  team  members:  the  ASAC,  ASWOC,  and  ASWFCG.  It  is 
here  that  the  real  challenge  of  accurately  and  realistically  renl.cing  or 
accenting  the  (human)  team  member  in  a  iynamioally  changing  envi-  ^nment 
will  be  met.  Some  form  of  knowledge-baited  simulation  is  perhaps  the  path 
to  suooess  in  this  area.  Simulating  realistic  communications  will  be  a  parti¬ 
cularly  important  area,  not  so  much  in  terms  of  how  to  generate  the  speech, 
as  ''n  formatting  appropriate,  human-like  messages.  Speeoh  recognition  will 
be  required  when  the  state-of-the-world  oannot  be  deduced  from  other  user 
inputs  suoh  as  NTDS  button  pushing.  (The  major  contribution  of  speech  recognition 
is  in  supporting  performance  measurement,  discussed  later).  The  trainee 
stations  are  currently  conoeived  to  be  based  upon  a  graphics  terminal  augmented 
with  a  traok-ball  or  joystick,  touch  entry  pad,  and  functional  keyboard. 
No  attempt  will  be  made  to  develop  "look-alike"  consoles  whioh  emulate  the 
operational  equipment.  Nevertheless,  the  stations  win  have  sufficient  fidelity 
so  that  operational  Navy  observers  or  users  will  not  require  a  quantum  leap 
of  imagination  to  see  the  appropriateness  of  the  system. 


1,  Note  that  this .statement  does  not  imply  that  team  member  A  must 
be  able  to  perform  the  job,  explicitly,  of  team  member  B;  but  rather 
that  member  A  must  be  able  to  compensate  for  inadequacies  of  member  B's 
performance.  Wagner  cites  George,  C.E.,  The  view,  from  the  underside:  Task 
Demands  and  Group  Structures.  Professional  Paper  11-6V,  Alexandria,  Virginia: 
Human  Resources  Research  Office,  196? 
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APPENDIX  B 

TEAM  TRAINING  RESEARCH  BACKGROUND 

A  review  of  the  team  traimug  literature  was  oonduoted.  Fortunately, 
there  were  exoellent  recent  reviews  already  in  existence  (e.g.,  Collins, 
1977}  Nieva  et  si.,  1978}  Popelka  and  Kerr,  1980}  Riaao,  1980;  and  Wagner 
et  al.,  1977).  The  objective  of  this  review  was  to  establish  a  research 
focus  for  the  development  of  a  research  devioe  specifically  containing  the 
emerging  technologies  of  adaptive  training,  automated  performance  measurement, 
and  computer  voioo  recognition  and  synthesis. 

This  review  generally  reinforoed  the  notion  that  team  training  research 
is  indeed  in  an  early  stage.  Consequently,  it  was  believed  appropriate  for 
this  review  to  emphasize  research  which  is  both  fundneetal  and  clearly  essential 
for  the  development  of  improved  team  training  methods.  For  example,  fidelity 
of  simulation  issues  were  believed  to  be  secondary  to  an  understanding  of 
team  phenomena  and  effective  training  methods;  furthermore,  suoh  fidelity 
considerations  probably  would  increase  the  research  system  complexity  and 
oost.  Some  researoh  issues  highlighted  in  the  literature  were  ignored  beoause 
these  would  require  sweeping  changes  in  Navy  procedures;  for  example,  the 
structure  of  Navy  teams  was  taken  aa  a  given,  since  change  would  be  required 
throughout  the  training  system,  the  rating  system,  and.  equipment  design. 

As  a  result  of  this  selection  process,  the  following  topics  were  derived 
from  the  literature  review  as  relevant  to  the  current  effort:  definition 
of  "team,"  communications,  feedback,  training  sequence,  environment,  and 
performance  measurement. 

DEFINITION  OF. TEAM 

Before  team  training  researoh  can  be  sensibly  structured,  one  needs 
to  define  the  term  "team."  On  the  other  hand,  there  is  a  dilemma  as  the 
definition  also  represents  a  theoretical  construct  requiring  research  for 
enrichment.  Furthermore,  since  most  related  research  has  been  performed 
on  “small  groups,"  there  is  a  need  to  also  consider  the  definition  of  "group." 
Collins  (1977)  has  performed  an  excellent  piece  of  work  comparing  the  literature 
in  these  two  research  areas. 

Rather  than  list  the  multitude  of  definitions  for  team  and  small  group, 
only  a  selected  few  will  be  presented,  as  the  commonality  is  greater  than 
the  differences.  Rizzo  (1979),  for  example,  chooser  the  definitions  of  Klaus 
and  Glaser  (1968): 

...a  team  is  usually  well  organized,  highly  structured,  and  has  relatively 

formal  operating  procedures— as  exemplified  by  a  baseball  team,  an  aircraft 

orew,  or  ship  oontrol  team.  Teams  generally: 
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•  are  relatively  rigid  in  structure,  organization,  and  communi¬ 
cation  networks, 

•  have  well  defined  positions  or  member  assignments  so  that 
the  participation  in  a  given  task  by  each  individual  can 
be  anticipated  to  a  given  extent, 

•  depend  on  the  cooperation  or  coordinated  participation  of 
ueveral  specialized  individuals  whose  adtivitieo  oontain 
little  overlap  and  who  must  each  perform  their  task  at  least 
with  some  minimum  level  of  proficiency, 

e  are  often  involved  with  equipment  or  tasks  requiring  poroeptual- 
motcr  activities, 

•  oan  be  given  specific  guidance  on  job  performance  baaed 
on  a  task  analysis  for  the  team's  equipment,  mission,  or 
situation. 

A  small  group,  on  the  other  hand,  rarely  is  so  formal  or  has  well- 
defined,  specialized  tasks— as  exemplified  by  a  Jury,  a  board  of 
trustees,  or  a  personnel  evaluation  board.  As  contrasted  with  a 
team,  small  groups  generally: 

•  have  an  indefinite  or  loose  structure,  organization,  and 
communication  network, 

•  have  assumed,  rather  then  designated,  positions  or  assignments 
so  that  eaoh  individual's  contribution  to  the  accomplishment 
of  the  task  is  largely  dependent  on  his  own  personal  charac¬ 
teristics, 

e  depend  mainly  on  the  quality  of  independent,  individual 
contributions  and  oan  frequently  function  well,  even  when 
all  or  several  members  are  not  contributing  at  all, 

«  arc  often  Involved  with  complex  decision-making  activities, 

e  oannot  be  given  muoh  specific  guidanoe  beforehand,  sinoe 
the  quality  and  quantity  of  participation  by  individual 
members  is  not  known. 

Wagner  et  al.  (1977)  adds  "teams  are  goal,  or  mission-oriented  and  so 
the  specific  oontext  in  which  the  team  will  operate  must  be  considered  before 
any  training  or  evaluation  technique  can  be  applied." 

Collins  (1977)  frames  his  discussion  in  terms  of  the  definition  by  Smith 
(1967),  which  includes: 
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(1)  the  largest  of  two  sets  of  two  or  more  individuals  jointly  charac¬ 
terized  by 

(2)  a  network  of  relevant  communications, 

v 

(3)  a  shared  sense  of  collective  identity t  and 

(4)  one  or  more  shared  goal  dispositions  with  associated  normative 

strength* 

Collins  notes  that  the  above  definition  of  group  is  considered  to  be 
of  maximum  use  because  its  elements  represent  a  ''critical  mass"  in  a  real 
empirical  sense.  There  should  be  a  sharp  increase  * "  the  probability  that 
grouplike  phenomena  will  occur  once  observable  configurations  of  events  satisfy 
the  four  definitional  requirements.  According  to  Zander  (1971)*  these  useful 
properties  include: 

1 .  awareness  of  accomplishment 

2.  feelings  of  satisfaction 

3.  stronger  desire  for  sucoess 

4.  groups  work  harder 

5.  coordinate  their  efforts  more  effectively 

6.  less  strain  on  interpersonal  relations 

7,.  wore  attracted  to  membership 

8.  group  becomes  more  productive 

It  would  appear  that  a  first  stage  in  team  training  should  address  the 
four  requirements  of  a  group,  to  ensure  establishment  of  a  strong  group, 
which,  in  turn,  should  lead  to  the  multiple  benefits  of  grouplike  phenomena. 

COMMUNICATIONS 

Upon  examination  of  ASW  teams  in  operation,  it  is  clear  that  interaction 
between  team  members  takes  two  direct  forms:  (1)  verbal  communications  and 
(2)  information  passed  through  oonsole  actions  resulting  in  display  changes. 

(A  third,  indirect  form,  is  the  effect  of  task  performance  by  one  team  member 
on  the  task  of  another  member.)  Communications,  therefore,  embody  the  team 
interactions,  and  are  obviously  critically  important  to  team  performance. 

Collins  (1977)  reports  that  several  work  conditions  (such  as  efficient 
routing  of  necessary  information  or  direct  and  rapid  access  to  information) 
have  a  positive  effect  on  croup  performance.  However,  Popelka  and  Knerr 
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(1980)  note  that  "Steiner  (1966)  applied  the  term  "process  loss"  to  the  degra¬ 
dations  of  team  productivity  produoed  by  individual  processes  and  he  empirically 
demonstrated  process  loss  effects"  (Steiner,  1972)  and  that  "oommur.i cations 
are  an  example  of  team  processes  that  diminish  team  performance."  Even  more 
strongly,  they  say  that  "the  most  consistently  demonstrated  effect  of  communi¬ 
cation  on  team  performance  is  that  the  extent  of  communication  is  inversely 
related  to  team  productivity1'  (Johnston  and  Briggs,  1968;  Meister,  1976; 
O'Brien  and  Owens,  1972:  Steiner,  1972). 

However,  Nieva  et  al.  (1y78)  point  out  the  results  are  mixed,  depending 
on  the  task  situation: 

1.  While  communi.-ax.ion  appears  to  have  positive  effects  on  problem 
solving  tasks,  the  opposite  was  generally  found  among  vigilanoe- 
monitoring  studies. 

2.  Steiner  and  Dodge  (1956)  found  that  communication  improved 
performance  in  unstructured  tasks  but  communication  had  no  effeot 
on  structured  tasks.  Also,  Thibault,  et  al.  (I960)  found  that  intragroup 
communication  is  especially  critical  with  unstable  task  demand; 
the  concept  of  stability  is  closely  related  to  structure. 

3.  Lanzetta  and  Roby  (I960)  found  that  the  number  of  requests 
for  information  was  negatively  related  to  performance  while  the 
ratio  of  volunteered  to  total  information  was  positively  related 
to  performance. 

Clearly,  communications  is  a  fundamental  team  research  area.  It  v/ould 
appear  that  the  passing  of  essential  information  between  team  members  is 
a  basic  requirement,  and  that  team  performance  will  be  diminished  if  these 
items  of  information  are  not  passed,  nuwever,  the  effioacy  of  additional 
information  is  a  moot  researoh  question. 

FEEDBACK 

"Performance  feedback  is  unquestionably  the  single  oritioal  parameter 
in  team  or  individual  training"  (Kanarick  et  al.  1971).  Klaus  and  Glaser 
(  1968)  conclude  "...it  appears  essential  that  team  practice  result  in  dear 
and  immediate  reinforcement  following  each  correct  team  response;  praotioe 
in  the  absence  of  team  reinforcement  for  criterion-level  performance  is  more 
likely  to  lead  to  a  decrement  in  team  proficiency."  Such  is  the  status  of 
feedback  about  trainee  performance  or  knowledge  of  results  (KOR).  There 
seems  to  be  general  agreement  on  this  point;  however,  beyond  this  not  muoh 
else  is  dear  with  regard  to  design  for  optimal  feedbaok. 

Hall  and  Rizzo  (1975)  list  some  of  the  problems  identified  by  Alexander 
and  Cooperland  ( 1 96 5 ) : 
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1.  There  may  be  several  criteria  of  effective  team  performance 
w  th  no  clearcut  tradeoffs  among  them.  These  criteria  may  be  vague 
and  difficult  to  state  objectively  and  may  chaDge  during  system 
operations. 

2.  In  order  for  a  team  to  operate  effectively,  it  is  necessary 
for  its  members  to  develop  and  maintain  individual  skills  as  well 
as  skill  in  working  together.  There  is  a  possibility  that  these 
skills  may  require  different  feedback  procedures  which  may  mutually 
interfe. 

3.  When  a  complex  system  operates,  there  is  usually  a  large  volume 
of  information  available  about  the  state  of  the  environment,  the 
state  of  the  system,  and  the  p-rformanoe  of  system  personnel.  Some 
of  this  information  may  be  conducive  and  some  inimical  to  effective 
learning. 

To  date,  the  research  on  feedback  or  KOR  has  served  to  shew  the  complex 
issues  involved  in  designing  optimal  feedback  for  team  training.  The  findings 
of  Briggs  and  Johnston  (196b),  as  briefly  summarized  below,  serve  to  make 
this  point. 

1 .  Use  of  individual-specific  KOR  is  desirable  when  one  team  member 
oannot  compensate  for  the  other. 

2.  Low-ability  personnel  may  benefit  from  either  individual  or 
team  KOR,  but  Mgh-abillty  personnel  may  benefit  from  individual 
KOR,  and  performance  may  deteriorate  under  team-speoifio  KOR. 

3.  Members  will  attempt  to  maximize  performance  about  whioh  they 
reoeive  specific  KOR;  furthermore,  this  improvement  may  be  at  the 
expense  of  other  characteristics  of  team  performance. 

M.  If  highly  specific  KOR  is  given  early  in  team  training,  it  would 
actually  interfere  with  skill  acquisition. 

Hall  and  Rizzo  (1975)  oonclude: 

Unfortunately,  many  relevant  questions  whioh  would  logically  arise 
in  attempting  to  apply  KOR  to  a  team  training  situation  are,  at 
best,  only  partially  answered  by  the  laboratory  researoh  in  this 
area. 

A  partial  list  of  suoh  questions  follows. 

1.  When  should  information  be  provided? 

2.  What  information  should  be  provided? 
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3.  Who  should  receive  feedback? 

1*.  How  should  KOR  be  provided? 

5.  Who  should  provide  KOR? 

6.  How  much  of  what  kind  of  KOR  should  be  provided  at  various  learning 
stages? 

Clearly,  feedback  is  an  area  requiring  emphasis  in  team  training  research. 
EMERGENT/ESTABLISHED  ENVIRONMENTS 

The  literature  treats  team  tasks  differently  depending  on  the  environment: 
a  "stimulus-response"  model  is  applied  where  the  tasks  can  be  almost  completely 
specified  and  team  assignments  are  fixed;  and  an  "organismic"  model,  where 
the  team  is  treated  as  an  organism  of  which  the  individuals  are  components, 
is  applied  when  the  individuals  have  considerable  discretion  in  the  manner 
tasks  are  performed  for  different  contingencies.  These  two  task  environments 
have  been  defined  by  Boguslaw  and  Porter  (1962)  as  follows: 

An  established  situation  is  one  in  which  (1)  all  aotion-relevant 
environment  conditions  are  specifiable  and  predictable,  (2)  all 
aotion-relevant  states  of  the  system  are  specifiable  and  prediotable, 
and  (3)  available  research  technology  or  reoords  are  adequate  to 
provide  statements  about  the  probable  consequences  of  alternative 
actions.  An  emergent  situation  is  one  in  whioh  (1)  all  aotion-relevant 
environmental  conditions  have  not  been  specified,  (2)  the  state 
of  the  system  doea  cot  correspond  to  relied-upon  predictions,  and 
(3)  analytic  solutions  are  not  available,  given  the  ourrent  state 
of  analytic  technology. 

Note  that  there  is  aotually  a  continuum  of  situations  in  whioh  the  end  points 
can  be  described  as  established  or  emergent. 

The  emergent/established  distinction  is  of  particular  importance 
here,  since  it  is  the  emergent  situation  which  a  number  of  investigators 
think  is  especially  appropriate  for  team  training.  Wagner  et  al.  (1977) 
notes  that  providing  skills  to  deal  with  emergent,  unstructured  situations 
is  seen  as  a  major  goal  of  team  training,  and  that  an  emergent  situation 
permits  a  more  realistic,  less  abstract  approach  to  th»>  investigation 
of  team  training  variables  than  does  the  established  situation.  Popelka 
and  Knerr  (1980)  state  that  "...team  training  in  communications  and  other 
interactive  skills  is  valuable  when  tasks  require  such  skills,  when  the 
work  situation  is  emergent,  and  when  task  requirements  are  highly  complex. 

One  concludes  that  emergent  situations  are  the  primary  task  environments 
for  team  training  research. 
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TRAINING  SEQUENCE 

The  moat  effective  method  of  sequencing  of  training  has  been  a  long¬ 
standing  issue.  One  of  the  more  prominent  Issues— whether  individual  training 
should  precede  team  training  or  the  oth«;r  way  around— will  not  be  treated 
here,  since  the  principle  of  individual  training  first  is  well  established 
in  the  Navy.  There  is,  of  course,  supporting  evidenoe  that  team  training 
given  before  individual  proficiency  has  been  achieved  may,  in  faot,  cause 
a  decrement  in  individual  proficiency  (Horrocks  *t  al.,  I960). 

Otherwise,  there  are  a  number  of  offers  of  guidance  in  this  matter  presented 
in  the  literature: 

Kanarick  (1971)  proposed  three  phases  for  team  training:  "Initially, 
there  is  the  need  to  train  individuals  in  the  procedural  aspects  of  their 
jobs,  doctrine,  and  the  process  approach  to  decision  making.  This  training 
should  be  followed  by  a  phase  in  whioh  team  members  are  instructed  as  a  unit, 
learning  the  Interactive  and  communicative  requirements  of  team  functioning. 
The  final  phase  is  devoted  to  tactioal  training  where  teams  are  taught  to 
apply  their  procedural  and  interactive  skills  to  certain  situations  requiring 
innovative  and  creative  behaviors." 

Jeantheau  (1969)  presents  four  basic  principles: 

1.  Training  objectives  are  specified  across  the  range  of  devioe 
capability. 

2.  Each  exercise  is  structured  and  controlled  with  respect  to  those 
specified  objectives. 

3.  Training  involves  a  systematic  progression  through  the  objectives 
according  to  levels  of  difficulty. 

4.  The  progression  through  the  sets  of  exercises  is  sensitive  to, 
and  guided  by,  measured  trainee  performance. 

With  regard  to  progression  through  levels  of  difficulty,  Hall  and  Rizzo 
(1975)  cite  the  work  of  Smode  (1972)  indicating  a  number  of  methods  to  manipulate 
problem  difficulty: 

1.  Easy  to  hard  continuum  (e.g.,  continual  increase  in  the  events 
provided) . 

2.  Procedural  to  fully  integrated  continuum  (e.g.,  later  exercises 
emphasize  full  utilization  of  portions  of  system  used  only  procedurally 
in  earlier  exercises), 

3.  Increasingly  stringent  conditions  of  performance  (e.g.,  winds, 
speeds,  etc.). 
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4.  Increasingly  stringent  error  tolerances  (e.g. ,  precision  demanded). 

5.  Amount  of  stimulus  support  (prompts,  cues,  KOR,  etc.). 

While  the  literature  is  iu  substantial  agreement  about  the  appropriateness 
of  sequencing  training  through  an  easy-hard  oontinuum,  the  actual  design 
is  somewhat  of  an  art  form.  It  appears  that  empirical  tests  of  alternative 
sequences  are  in  order,  circumstances  permitting. 

PERFORMANCE  MEASUREMENT 

The  requirement  to  provide  feedback  and  tt  .sequence  training  based  on 
performance  leads  directly  to  the  requirement  for  a  performance  measurement 
system.  As  Hall  atd  Rizzo  (1975)  specify:  "What  is  needed  for  team  training 
is  an  objectively  based  performance  measurement  system.  This  should  consist 
of  performance  standards  and  means  for  obtaining.,  processing,  and  comparing 
student  'scores'  to  the  standards  for  arriving  at  summary  evaluations  and 
for  giving  student  feedback  on  hew  well  he  is  doing  during  training." 

However,  while  individual  and  collective  performance  may  lead  directly 
from  systems  analysis  methods,  measures  of  team  phenomena  do  not.  Simply 
stated,  we  do  not  know  what  is  accomplished  during  team  training.  Hall  and 
Rizzo  (1975)  comment  that  "While  everyone  professes  intuitively  to  be  able 
to  recognize  a  good  team— the  'I'll  know  it  when  I  see  it'  phenomena— no 
one  seems  able  to  articulate  its  dimensions  with  sufficient  clarity  to  permit 
the  development  of  training  procedures  for  producing  it."  And  further: 
"Apparently,  the  operational  squadrons  feel  that  the  individuals  sent  to 
them  from  the  team  training  program  are  not  yet  ready  tr.  function  wiv.hin 
a  team,  it  is  felt  that  'something'  has  not  been  achieved  that  is  needed 
for  effective  team  functioning." 

In  view  of  this  uncertainty  about  quantifiable  aspects  of  team  phenomena, 
it  would  appear  that  exploratory  research  is  in  order  to  search  for  meaningful 
measures. 
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APPENDIX  D 

HIGH  LEVEL  KNOWLEDGE  0?  THE  ASWOC 

In  order  to  develop  a  knowledge  representation  architecture  and  oontrol 
struoture  for  the  team  member  models,  it  was  neoeasary  to  articulate  a  rep¬ 
resentative  sample  of  the  lovlrdge  and  rules  used  in  the  ASW  problem.  For 
tnis  purpose,  the  high  level  rules  and  knowledge  employed  by  the  ASWOC  during 
the  soenano  was  assembled  and  expressed  in  an  "if/fcnen"  format  analogous 
-o  production  rules.  Seme  of  the  conditions  described  in  the  rules  cun  come 
about  at  any  time,  and  the  rules  relating  to  these  conditions  have  been  described 
in  table  format  in  Table  D-1 .  Other  conditions  can  be  organized  sequentially 
and  the  related  rules  are  shown  in  Figure  D-1 .  The  numbers  above  some  of 
the  boxes  in  this  figure  refer  to  the  "floating"  rules  given  previously  in 
Table  D-1  which  are  likely  to  be  invoked  at  that  point  in  the  sequence. 
The  dashed  lines  in  the  figure  represent  branches  tfhioh  should  not  be  taken 
in  the  training  ooenario. 

It  should  be  emphasized  that  the  rules  shown  in  the  table  and  figure 
are  relatively  high  level  rules  wb4  'h  require  further  knowledge  to  evaluate 
and  carry  out.  Therefore,  racr.  -f*  on  the  figure  might  aotually  involve 
invoking  many  production  rules  to  evaluate,  and  each  "then"  may  require  many 
rules  to  accomplish.  To  illustrate,  one  box  (shown  with  bold  outlines  on 
page  D-H)  was  arbitrarily  chosen  and  analyzed  in  more  detail.  As  shown  in 
Figure  D-2 ,  the  phrase  "assume  the  duties  of  the  attacking  ship"  used  io, 
this  box  requires  invoking  many  rules  to  acocoplish. 

This  analysis  was  used  to  aid  the  software  designers  in  gaining  an  under¬ 
standing  of  the  quantity  of  information  in  the  form  of  rules  and  procedures 
whi.'h  the  team  member  models  will  have  to  employ,  as  well  as  the  variety 
of  conditions  whioli  the  models  will  have  to  monitor. 
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TABLE  D-1.  "FLOATING  PRODUCTION  RULES*  WHICH  THE  ASWOC  MAX  EMPLOY 
AT  ANY  TIME  DURING  THE  SCENARIO. 


Number 

IF 

THEN 

1 

The  submarine  is  detected  within 

6000  yards  of  any  friendly  unit 

Conduct  urgent  attack 

2 

The  ship  is  maneuvered 

Report  maneuver  to  the  SAC 

3 

A  weapon  is  prepped 

Report  weapon  prep  to  SAU 

k 

It  is  ueoessary  to  adjust  sectors 

Adjust  sectors 

5 

The  range  and  bearing  to  the 
oontact  is  known 

Report  to  SAU  at  least  once 
per  minute 

6 

The  course  and  speed  of  the 
submarine  is  known 

Report  tc  SAD  and  all  stations 

7 

A  weapon  is  fired 

Report  to  SAD  with  firing 
bearing 

8 

The  ASWOC  As  going  to  communicate 
externally 

Select  radiotelephone  position 
on  the  console  oommuni cation 
panel,  depress  footkey,  talk 

9 

The  ASWOC  is  going  to  communicate 
with  the  ASWFCO 

Select  the  1JS  position  on 
the  oonaole  communications 
panel,  depress  the  feotkey, 
talk 

10 

The  ASWOC  is  going  to  communicate 
with  the  AS AC 

Select  interphone  position 
on  the  console  coamuni cations 
panel,  depress  the  interphone 
button  for  ASAC„  wait  for 
an  acknowledgement,  depress 
footkey,  talk 

11 

A  track  symbol  or  point  is  to  be 
hooked 

Depress  ball  tab  enable, 
roll  the  ball  tab  over  the 
symbol  or  point,  depress 
hock  FAB,  wateh  for  the  oirole 
to  appear  around  the  symbol 
or  point  and  the  DRO  to  change 

I  •  If 
i  i 

r*  *T 

r - ^ - , 

I  I 

.  Tha  Contact  Don  not  Show  . 
Soon®  Serial  Stranyth  and  ! 
1  Don  not  Ti».h  Lika  a  Sub  I 


NOTE;  *  lnd«a*a  floating  Production  Rube 


I 

|  THEN 


!  I  Am  Goins  to  CUHify  It  J 

;  Non  Sub  and  Raturn  to  ' 

I  Satrtb  I 

I _ 1 


/ 
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7 h«  Ship  hr ;  no  Long 
VKupom.  «  Sonar  Oindittom 
do  rai  A'kjW  kV.  to  (fold  thi 
Con-eel  at  Looj  (tings 


l(  I  CUully  thi  Contact  *1 
Poiuib  ContuuncO  Lnvol 
High  or  Higher 


!  \,-,i  Going  to  Mike  tin 
(Jed.  on  to  Enornte  the 
Gh.icfc  Clin 


I  Detide  to  t*«ojte  tni 
Attack  I’Un 


1  Wi»i  C.  ■•••• 
Pbn,  3A  <•* 

*h  Attack 

S 

IF 

-y 

z 

Thi  Sub  it  D*tieti:il  it  Long 

1  |  Deddi  not  to  Exnculi  J 

I  tt*  Adxdt  Plan 

I  _ j 


Allow  M*  to  Hold  Contact 
•t  Long  Hungt 


I - O 


1  THFN 

t _ i. _ j 

l  |  will  Continue  to  Track  | 
|  ih*  CfMltACl  I 

L _ _ J 


Figur*  D-1.  High  Lw«i  Kr>owtid*i  of  thi  A8WOC 
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I  Auumt  th«  Dutkn  o! 
Attacking  Ship 


I  Am  Going  to  Dotarmini 
Typ«  Weapon,  Bloodhound 
of  Baaattt 


^iV"i.i-.  ft.  .  , 


.■,nT^T,»rj,i  i  *,i  p.i  f  w  ^prjrd-T-  •• 
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I  A  Good  Solution  i*  ttriuhtitll 
|  or  th*  1‘Uno')  lUuuwi  Glew  1 


\k  Not  Good 
^•nga  it  Not 


THEN  l  1  A*n  yoiny  to  Aiivtupl 

. to  Ci  it  u  Good  Solution 

I  or  to  Ol&nr  thd  thiiiuti 


p  Vtrity  t>*  |p 
llolutton  And  r — 


_ i_ 


Th*  Solution  li  Good  THFfl .  1  Am  Going  to  Order  tU 

•nd  trio  Fl*i>g»  ii  clour  ““ r  Wwpon  Fin'd 


- 1 


n  iu tu 

I  Ord*r  thn  Weapon  F»(rf  - 


1  I  Turii  1  Ain  Going  to  Ortltf  f. 

^  I  Ord*r  thn  Weapon  hacrt  U — —  ft.  Mauuavu  to  Clgat 

j  ;  Weapons  Danya  Ares 


•aw _ X..,/ 


i.wwe  Out  ot  tiia  J21H1  [t 
pom.  IViv*  Anti 


Ellt.lit  AtisiJt  I 

to  Ota  foil'd , 
Going  to  | 

tilt  SSiip  | 

IF 

"  -**■ - 1 


ttrt  Ai  in  Ship 


;  I  Am  Going  to  Avbimpt  to  j'  I  "  . "1 

|  Hold  Contact  on  th»  "forget  j  |  1 

EM  •«*  Wltllln  th*  (  |  |  Atom*  tin.  Dm.no  ol  I 

— rjBxtof  Aidgnod  MoBy’  I  ,  Atttclung  Mop  1 

I  tli*  Attacking  Ship  to  Kwp  j  I 

I  Clear  ot  th*  Attacking  Ship  j  ■  j 

>L  '-tV 

11  I  |,r 

r _ _5vT- _ ^  *8* _ ]_[ _ ( 

i  i  r  ”i 

I  Th*  Attaching  Ship  Low*  |TH£N  -  j  t  Am  Going  to  Aaumt  th*  I 
|  Contact  |  ^1  Dull**  ot  Attacking  Ship  I 

'  I  I 


i  I  Av.tim*  tin.  Omni  u! 

|  Atttcltinu  tUup 

( 

I - - - 
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Iht  Court*  Choi»n  Will 
Not  Ktop  thfl  ConUft 
Clati  ot  tht  UbIIIb 


I  Turn  Utt 


7s: 

IF 


I  Am  doing  to  Turn 
Lift,  Bight  or  Contlnut 
Ft  tout  Court*  . 


The  Choton  Court*  Will 
Kwp  tit*  Contact  Clc*r 
ol  tit*  B«r!l*t 


|  ll  Am  Going  to  Inlotm  Sunn  I 

l  THEN  0th*1  1  A'n  tiom»  D"9  th,l _ 

u  “  ►^Contact  Thru  thj  B,Hltt  *nd| 

I  Iwhy  or  Ad)utt  tht  Count  I 


Th*  Cht 
Kttp  th 
ol  tilt  E 


1  Am  Going  to  Cnooi*  * 
Court*  ri'tt  trill  not  put 
th*  Contact  hi  th*  UnHko 


- * — — - — 

-on,.  1  Am  Going  to  R*ruH  Wtttr 

AND 

1  Continue  Prciant  Court* 

bill  Tim* 

- 1 - ST  “ 

Httkrt  to  Optimum  Hang*  to 
Hold  Contact  from  tht  Wtttr 
I  Entry  Folnt 


I  Am  doing  to  Choot*  ■ 
Count  that  will  not  put 
th*  Contact  In  th*  6*HI*t 


Th*  Chot*n  Court*  will 
kt*p  th*  Contact  Cltar 
ot  th*  BtHltt 


_ NZ _  r - 1  I 

f-  I  1  I  Am  QoHit  to  Intorm  .  | 

l  Th.  rnima  Thrum  will  l™«,  I  lorar  that  I  Am  doing  »  I  AND. 

i  JESS*1,  _J 

1 _ —  —  -*  ' - 


NOTk:  •  India  tat  Rotting  Froduetkm  Rultt 


Flfur*  0-1.  High  LavW  Kiwdtdg!  «H  th*  A8WOC,  oorrtlnuwt 
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“1 


'  It  I  iio  not  get  »i>  up  Suu.i !  <_HfN 
'  Report  | 

L - ^ - J 

"if 

I  I  IF 

- u - - 


l  Am  Going  trj  Become 
Aurrt  £Uiip 


thi1 


L. 


•10 


I 


I  THEN  j  I  Am  Going  to  gel  •  Stele 
I  Order  i  VF.CTAC  j - ►»  end  Sunt  Report  Irom  the 


r—  — 

|t  the  Iretk 


If  -A~ 

r  s  r* 

f.  !  !if 


I  Depreei  tlte  Engege  ASW  THFN 
A/C  end  Verbelly  Order  |  — 
the  Engagement  | 

_ I 

*10/11  J  JUF _ 

.  M  Mer  I  ft  r-e*  ****  j 

Iruew  I'  *m  Ootn*  10  ‘,00„k  th# 
with  the  THEN«  J  Truck  end  Uetueu  Fnc»ge  | 
with  the  j. —  ASW  A/c  yAB  ond  vtIt»llY  . 

|  |  Order  the  Engagement  J 


I - — 

V 

'IF 

•10  jl  _ 

i - 


I  Am  Going  to  Order  e 
Vectec 


| -  1 


Mm  to  En»ge  the  |_OB  J  I  Am  Oolng  to  Ahum  H  » 

tfththe  Helicopter  ^  Dutle*  ui  Aulit  Ship 


I _ 


_ 1 


I _ 


|  ASAC 

I _ 


f 


n~ 

I  nr 

J  *7 

.u.  - 


_ J 


•10 


ItHEN  I  Am  Going  to  Give 
I  |  Get  en  Up  St.tut  Report  IT-  J±\  Weepont  Free  Order 
!  |  |  the  ASAC 

1 _ I  I - 


the 

to 
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:  Thi  ASWFCO  Will  V«h»ll(* ) 

I  Adtnowhdgt  thtt  lw  hit  j 
i  mid#  thi  Amynnwcrt  [ 

•  r“J 

!  AND 

j  1  1 

j  Thi  Anijnirwm  Bw  will  l 

•  Aopmt  on  th<  Symbol  j 

|  |  a  _  |  _ _  |  HHl  j 

T 

’  A  ilWN 


I  0«i»  mi  thi  Atttfn  Mk 
116  VAB  ind  VwhiHy 
Order  thi  AtJlgnmint 


Thi  ASYVFCO  will  Acourt 
thi  Awignmim 


Thi  ASWFCO  Auism  tin  THEN 
MK  It#  to  tin  Traill 


1  Am  Going  to  Hook  «!» 
HiUooptir  Symbol 

. 

I  Thi  Cineil  Aolgnmirtt  Alert  | 
jwlll  bi  Mitt  to  thi  ASAC  j 

* . . -J 

IF 

\7  . 

t 

‘  AND 

1 

1  Hook  th»  HiUooptir 
Symbol 

1  Am  Going  to  Doptttt  ttw 
THEN  w  CinoM  VAB  ond  V wholly 
"  "■**  Uniw  thi  ASAC  to  Clow 

the  Hilo 

—  n - 

I  Order  thi  ASAC  to  Ooar 
thi  Hilo 


NOTE:  •  lr>*K*t»  Fto*T*n*  Fre-Netloo  FluK* 
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Am  Going  to  Hook  the 
elicopter  Symbol 


I  Hook  tho  Helicopter 
Symbol 


I  The  Cental  Atslgnntciil  Alert  | 
jwlll  bj  snrt  to  the  ASAC  j 


|  Am  Going  to  Dept  in  the 
Cancel  VAB  mid  Verbelly 
Order  the  ASAC  to  Cle» 
the  Hilo 


—  — , 

1  Order  the  ASAC  to  Clror 

|  Am  Going  to  Ask  the 

TT1EW-fc.  ASWFCO  for  Couree  end 

the  Halo 

w  Wcupont  Roraommendetlone 

L - n - 

|  Ask  lor  Recommendations 


I  Am  Going  to  Receive  Rec- 
commendations  horn1  the 
ASWFCO 


Determine  Type  Wmpon. 
B1  odliound  or  Benett 


Flfur*  0-1  Amplification  of  ASKVOC  KnowtedB*.  continued. 
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APPENDIX  E 

TRAINING  CONSOLE  CAPABILITIES 

In  order  to  perform  their  tasks,  the  members  of  the  ASW  team  must  utilize 
the  NTBS  console  and  the  ASWFCO  must  also  utilize  a  separate  weapons  oontrol 
panel.  While  look-alike  consoles  are  not  necessarily  required  for  the  ASW 
team  training  system,  the  funotlons  provided  by  the  consoles  must  be  available. 
For  this  reason,  a  description  of  the  relevant  funotlonal  characteristics 
of  these  consoles,  derived  from  the  Systems  Operator's  Manual  for  DD.963  Class. 
PD(A)-06166,  is  provided  in  the  paragraphs  which  follow.  Because  the  scenario 
proposed  for  the  initial  implementation  focuses  upon  one  aspeot  of  ASW,  only 
a  subset  of  the  console  capabilities  will  be  required.  These  required  capabil¬ 
ities  are  noted  in  the  descriptive  tables  which  follow. 

THE  NTDS  CONSOLE 

In  general,  the  NTDS  console  serves  a  variety  of  funotlons:  it  provides 
a  radar  display,  and  display  of  digital  information  on  the  digital  read  out 
(DRO).  In  addition,  the  operator  oan  use  the  various  buttons  to  request 
information  and  perform  calculations,  and  to  update  the  data  base. 

The  NTDS  oonsole  has  a  limited  number  of  buttons  which  are  used  to  provide 
a  very  large  range  of  funotlons.  To  accomplish  this,  the  oonsole  provides 
a  few  fixed  action  buttons  (FABs),  whloh  always  have  the  same  effect,  and 
a  set  of  variable  action  buttons  (VABs)  whose  function  depends  upon  which 
"array"  or  set  of  button  meanings  is  activated.  In  addition,  the  Digital 
Data  Entry  Unit  (DDEU)  allows  numerical  input. 

The  fixed  action  buttons,  shewn  in  Table  E-1 ,  and  the  DDEU  inputs,  shown 
in  Table  E-2,  are  used  by  all  three  team  members.  The  different  variable 
action  buttons  used  by  each  team  member  are  described  in  the  paragraphs  whioh 
follow. 

ASWOC  CONSOLE  FUNCTIONS 

The  ASWOC' s  mode  selection  capability  is  shown  as  an  array  tree  in  Figure 
E-1.  Of  tne  possibilities  shown,  the  training  system  soenario  requires  that 
the  ASWOC  use  the  ASWOC  primary  array  (shown  simply  as  "ASWOC"  in  the  figure), 
the  ASWOC  alternate  array  ( ALTN  ASWOC),  and  tKe  ASW  data  array  (ASW  DATA). 

The  ASWOC  primary  array,  shown  in  Figure  E-2,  is  used  to  assign  weapons, 
engage  targets,  oanccl  assignments,  and  perform  other  control  functions. 
A  brief  description  of  the  function  of  the  VABs  in  this  array  is  given  in 
Table  E-3.  In  this  and  in  the  following  tables,  the  VABs  needed  in  the  training 
scenario  are  marked  with  *n  asterisk. 
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The  ASWOC  alternate  array,  shown  in  Figure  E-3 ,  la  used  to  gain  additional 
Information  and  to  update  t^e  data  base.  A  brief  functional  description 
of  the  VABs  in  this  array  is  provided  In  Table  B-4„ 

Ti  ft  Aik  data  .array,  shown  in  Figure  E*4,  is  used  to  acquire  information 
ie  aid  the  decision  making  procesa.  A  description  of  these  VABs  is  provided 
in  Table  E-5. 
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TABLE  E-1.  FIXED  ACTION  BUTTONS  AVAILABLE  TO  ASWOC,  ASAC,  AND  ASWFCO. 


Function 


Array  Select 
Array  Sequenee 
Sequenoe 
Enter  Offset 


Pointer* 


Drop  Track 
Track  Number 
Height 

Rolling  Bali  Tab 


Ball  Tab 


Ball  Tab  Enable 


Center 


*  Not  available  to  the  ASAC. 
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TABLE  E-2.  SENSOR  CODES  FOR  DDEO  ENTRT. 


DDEO 

Value 

Senaor 

5 

1 

Ownahlp  Active  Sonar  Channel  1 

5 

2 

Ownahlp  Active  Sonar  Channel  2 

5 

3 

Ownahlp  Passive  Sonar  Channel  1 

5 

4 

Ownahlp  Passive  Sonar  Channel  2 

5 

5  • 

Remote  Ship  Active  Sonar 

5 

6 

Remote  Ship  Passive  Sonar 

5 

1 

Helo  Active  Sonar 

5 

8 

Helo  Passive  Sonar 

M/5 

9 

Active  Sonobuoy 

4/5 

10 

Passive  Sonobuoy 

4/5 

11 

Visual 

4/5 

12 

Radar 

4/5 

13 

MAD 

4/5 

.  14 

LOPAR 

4/5 

15 

Intelligence 

4/5 

16 

ESM 

4/5 

17 

CASS 

4/5 

18 

DIPAR 

4/5 

19 

DICASS 

4/5 

20 

Ownship  Sonar  Unit  31 

4/5 

21 

FLIR 

140 


ft 


NAVTRAEQUIPCEN  80-C-0095-1 


TABLE  E-2,  SENSOR  CODES  FOR  DDEU  ENTRY,  CONT 


DDEU  Value  Sensor 


4/5  22  Vide band/ Aoouatlc 


a  a- 
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TABLE  E-3.  ASWOC  PRIMARY  ARRAY  FUNCTIONS. 


l 


IF 


. 


t 


TAB 

Label 

Funotiioc  Summary 

1* 

ASSIGN  ASV  AC 

Orders  ASAC  to  assign  an  ASV  airoraft  to  the 
track  in  olose  control. 

2* 

ASSIGN  MK  116 

Orders  ASVFCO  to  assign  the  track  in  close 
oontrol  to  the  Mk  116. 

3* 

HOLD  FIRE 

Alerts  the  ASVFCO  and/or  ASAC  to  bold  fire 
on  the  track  in  olose  oontrol. 

H* 

ENGAGE  MK  116 

Orders  ASVFCO  to  engage  the  traok  in  olose 
oontrol  with  a  weapon  of  his  oholoe. 

5* 

ENGAGE  ASV  AIC 

Orders  ASAC  to  engage  the  traok  in  olose  oontrol 
with  a  weapon  of  his  oholoe . 

6 

CANTCO 

Notifies  SVC  that  ASVOC  oannot  oomply  with 
an  engagement  or  assignment  order. 

7* 

CANCEL 

Terminates  an  assignment  ourrently  in  progress. 

8* 

BREAK  ENGAGE 

Orders  ASVFCO  or  ASAC  or  both  to  break  all 
engagements  against  the  traok  in  olose  oontrol,, 

9 

9 

Not  implemented. 

10 

10 

Not  implemented. 

11* 

DESIG  SONAR  UNIT 

Orders  a  sonar  to  searoh  a  designated  position 
for  a  contact. 

12* 

USE  PRED  TRACK 

Orders  use  of,  or  termination  of,  predicted 
traok  data  for  a  oontaot. 

13 

13 

Not  implemented. 

U» 

in 

Not  implemented, 

*  Function  required  in  the  training  soenario. 
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TABLE  E-3.  ASWOC  PRIMARY  ARRAY  FUNCTIONS,  CONT. 


VAB 

Label 

Function  Sumary 

15 

15 

Not  implemented. 

16 

16 

Not  iapleaented. 

17 

17 

Not  implemented. 

18 

18 

Not  implemented. 
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ENTER 

MISSILE 

ASSOC/ 

DISASOC 

MANUAL 

TMA 

• 

COURSE 

• 

NEW 

TRACK 

VAB  NO.(V) 

0 

0 

0 

CD 

CD 

SENSOR 

• 

TRACK 

HISTORY 

SPEED 

• 

REPOS 

0 

0 

0 

1 9 

0 

© 

COMP 

LOFAR 

FIX 

# 

SONO- 

BUOY/ 

TIME 

* 

_ i 

TRACK 

DEPTH 

• 

DATA 

XMIT 

SONO- 

BUOY 

RF  CKN* 

TRANS¬ 

DUCER 

DEPTH* 

0 

0 

0 

0 

^  0  J 

*lndicatM  •  DDEU  antry  accompaniat  tha  VAB  action. 


Figure  E~3,  Alternate  ASMOC  array. 
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TABLE  E-4.  ASWOC  ALTERNATE  ARRAY  FUNCTIONS. 


VAB 

Label 

Funotion  Summary 

1 

1 

Not  implemented. 

2 

ENTER  MISSILE 

Enters  a  new  hostile  missile  traok  at  the  ball 
tab  coordinates,  and,  if  a  traok  is  in  olose 
oontrol,  ohanges  that  parent  to  a  missile  platform. 

3* 

ASSOC/DISSOC 

Associates  or  disassociates  two  specified  OSS 
traoks. 

4 

MANUAL  TMA 

Initiates  a  manual  TMA  override  for  an  OSS  track 
or  terminates  manual  TMA  override  for  an  OSS 
traok  or  PK  traok. 

5 * 

COURSE 

Enters  a  manual  value  for  oonrae  on  the  speoified 
UFCS-reported  (OSS  Mk  116  manual  subsurface) 
traok  or  ownship  when  speoified. 

6 

NEW  TRACK 

Manually  enters  a  Mk  116  manual  subsurf aoe  traok 
at  the  speoified  coordinates. 

7 

7 

Not  implemented. 

8 

8 

Not  implemented. 

9* 

SENSOR 

Enters  or  ohanges  the  sonar  source  for  a  speoified 
surfaoe  or  subsurfaoe  traok. 

10* 

TRACK  HISTORY 

Displays  traok  history  points  for  speoified 
OSS  traoks  or  ownship  (max  2). 

11* 

SPEED 

Enters  a  manual  value  for  speed  on  the  speoified 
UFCS-reported  (OSS  or  Mk  116  manual  subsurfaoe) 
traok  or  ownship. 

12* 

REPOS 

Repositions  the  speoified  UFCS-reported  (OSS 
or  Mk  116  manual  subsurfaoe)  traok  to  the  position 
of  the  ball  tab. 

•  Funotion  required  in  the  training  soenario. 
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TABLE  E-4.  ASWOC  ALTERNATE  ARRAY  FUNCTIONS,  CONT. 


[ 

f 

f 

i 

?; 

VAB 

Label 

Funotion  Summary 

i 

13 

COMP  LOFAR  FIX 

Displays  or  deletes  a  comparative  LOFAR  fix. 

1. 

>• 

| 

'l 

\ 

't 

U 

S0N0BU0Y  TIME 

Enters  a  sonobuoy  special  point  and  associated 
data  of  type,  time  remaining,  and  holding  or 
not  holding  contact. 

!; 

( 

{ 

\ 

15* 

TRACK  DEPTH 

Enters  a  manual  depth  or  use  doctrine  value 
for  depth  (440  ft.)  on  the  specified  UFCS-reported 
(OSS  or  Mk  116  manual  subsurf aoe)  traok. 

| 

i 

16 

DATA  XMIT 

Initiates  LINK-11  transmissions  of  a  specified 
OSS  track  or  looal  acoustic  bearing  line. 

3 

1 

! 

1 

17 

SONOBUOY  RFCHN 

Enters  the  radio  frequency  (RF)  channel  for 
a  specified  sonobuoy. 

t 

1 

18 

TRANSDUCER  DEPTH 

Enters  the  transducer  depth  of  a  specified  sonobuoy. 

i 

« 

Funotion  required  in  the 

training  scenario. 
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Figure  E-4.  ASV  data  array. 
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TABLE  E-5.  ASW  DATA  ARRAY  FUNCTIONS. 


VAB 

Label 

Function  Summary 

1* 

DISPLAY  SPA 

Request  or  delete  the  display  or  submarine's  probability 
area. 

2* 

DISPLAY  LLSA 

Request  or  delete  the  display  of  limiting  lines 
of  submerged  approaoh. 

3* 

DISPLAY  FOC 

Request  or  delete  the  display  of  the  furthest  on 
circle. 

4* 

DISPLAY  TDZ 

Request  or  delete  the  display  of  the  torpedo  danger 
zone. 

5* 

DISPLAY  WPNS 
LOAD 

Sequentially  displays  the  weapon  load  of  eaoh  oell 
of  the  ASROC  launoher  and  the  OTS  torpedo  tubes. 

6* 

6 

Not  implemented. 

7 

AIR  DENSITY 

Enters  and  displays  in  the  DRO  the  value  of  the 
air  densitv  to  be  used. 

8** 

LAYER  DEPTH 

Enters  and  displays  in  the  DRO  the  value  of  layer 
depth  to  be  used. 

9 

RVua  Limit 

Enters  and  displays  in  the  DRO  the  value  of  RVua 
limit  to  be  used. 

10* 

DISPLAY 

OCEAN  DATA 

Request  display  of  the  OCEAN  DATA  in  the  DRO. 

11* 

SONAR  UNIT 
STATUS 

Request  display  of  the  SONAR  UNIT  STATUS  in  the 
DRO. 

12* 

UFOS  STATUS 

Request  display  of  the  ASW  SYSTEM  STATUS  in  the 
DRO. 

*  Function  required  by  the  ASWOC  and  ASWFCO  in  the  training  scenario. 

**  Funotion  used  by  the  ASWFCO  in  the  training  scenario. 
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TABLE  E-5.  ASW  DATA  ARRAY  FUNCTIONS,  CONT. 


VAB 

Label 

Function  Summary 

13• ** 

BOTTOM  DEPTH 

Enters  and  displays  in  the  DRO  the  value  of  bottom 
depth  to  be  used. 

m** 

BOTTOM  TEMP 

Enters  and  displays  in  the  DRO  the  value  of  bottom 
temperature  to  be  used. 

15 

SALNTY 

Enters  and  displays  in  the  DRO  the  value  of  salinity 
to  be  used. 

16 

16 

Not  implemented. 

17 

DEEP  SOUND 
SPEED 

Enters  and  displays  in  the  DRO  the  value  of  DEEP 
SOUND  SPEED  to  be  used. 

18 

SURFACE 

SOUND  SPEED 

Enters  and  displays  in  the  DRO  the  value  of  SURFACE 
SOUND  SPEED  to  be  used. 

•  Function  required  by  the  ASWOC  and  ASWFCO  in  the  training  scenario. 

•*  Function  used  by  the  ASWFCO  in  the  training  scenario. 
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AS AC  CONSOLE  FUNCTIONS 

The  ASACs  node  aeleotion  capability  la  shown  as  an  array  tree  in  Figure 
E>5.  Of  the  possibilities  shown,  the  training  ays  ten  scenario  requires  that 
the  AS  AC  use  the  AS  AC  primary  array  (shown  simply  as  "ASAC"  in  the  figure), 

the  ASAC  alternate  array  (/.LTN  ASAC),  and  the  ASK  points  array  (ASH  POINTS). 

( 

The  ASAC  primary  array,  shown  in  Figure  E-6,  is  used  to  control  ASU 
1  aircraft  and  to  autonatloally  transfer  information  about  the  progress  of 
the  aiforaft.  These  functions  are  described  in  Table  E~6> 

i 

The  ASAC  alternate  array,  shown  in  Figure  E-? ,  allows  the  ASAC  to  reoeive 
recommendations  from  the  oomputer  oonoerning  the  control  of  the  ASH  aircraft. 

!  These  funotlons  are  desoribed  in  Table  E-7. 

The  ASV  points  array,  shown  in  Flgrre  E-6,  is  used  for  the  entry  of 
information  suoh  as  MAD  oontaots  and  radar  sinker  data.  The  functions  of 
these  VAfis  ore  desoribed  briefly  in  Tuble  E-8. 
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Figure  E-5.  ASAC  array  tree. 
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Figure  E-6.  ASAC  primary  array. 
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TABLE  E-6.  ABAC  PRIMARY  ARRAY  FUNCTIONS. 


VAB 

Label 

Function  Summary 

1* 

ASSIGN  ASW  A/C 

Assigns  an  ASW  A/C  to  a  target* 

2« 

ENGAGE 

Engages  a  target  with  an  ASW  A/C. 

3* 

WEAPON 

DPLOYED 

Indicates  that  a  weapon  has  been  deployed  by  an 
ASW  A/C. 

4 

4 

Not  implemented. 

5* 

ACCEPT  0/S 
CONTROL 

Designates  a  traok  or  ownship  controlled  ASW  A/C. 

6 

CANTCO 

Responds  with  cannot  oomply  to  an  ASSIGN.  ENGAGE 
or  ASSUME  A/C  control  alert. 

7 

CANCEL 

Cancels  the  assignment  of  an  ownship  controlled 
ASW  A/C  with  a  target. 

6* 

BREAK  ENGAGE 

Breaks  the  engagement  of  one  or  more  ASW  A/C  with 
their  targets. 

9 

ASSUME  A/C 
■  CONTROL 

Orders  a  PU  to  assume  oontrol  of  an  ownship  controlled 
ASW  A/C. 

10 

WEP  OFFSET 

Modifies  or  deletes  the  modification  to  the  WEP 
calculation. 

11 

DELETE  0/S 
CONTROL 

Deletes  the  designation  of  a  track  as  an  ownship 
controlled  ASW  A/C. 

12* 

REPOS 

Changes  the  position  of  a  track  without  affecting 
track  velocity. 

13 

TARGET  KILLED 

Notifies  SWC,  ASWOC,  and  Bridge  that  a  target 
was  destroyed. 

NAVTRAEQUIPCEN  80-C-0095-1 


TABLE  E-6.  ASAC  PRIMARY  ARRAY  FUNCTIONS,  CONT. 


VAB 

Label 

Function  Summary 

14* 

TRIAL  MANVR 

Requests  the  oaloulation  and  display  of  a  collision 
intercept  between  a  track  and  another  location. 

15* 

WAYPOINT 

Requests  modification  of  a  Trial  Maneuver  to  go 
through  a  specified  point  on  the  way  to  the  intercept 
point. 

16* 

STEADY  ON 

Indicates  that  a  traok  in  command  traok  status 
is  at  ordered  heading  and  at  ordered  speed. 

17* 

CMD  TRKNG 

Enables  or  disables  command  tracking  for  an  ownship 
controlled  ASW  A/C. 

18 

POS  COR 

Enters  a  corrected  position  for  a  traok. 

•  Function  required  in  the  training  soenario. 
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TABLE  E-T.  ASAC  ALTERNATE  ARRAY  FUNCTIONS. 


VAB 

Label 

Function  Summary 

1* 

VOICE  CALL 

Enters  or  ohanges  a  voice  call  for  an  ownship 
controlled  ASW  A/C. 

2* 

A/C  TYPE 

Enters  or  ohanges  the  A/C  type  for  an  ownship 
controlled  ASW  A/C. 

3* 

WILCO 

Response  to  assume  A/C  control  and  salvo  orders. 

4 

HELO  DIP  MANVR 

Initiates  a  helo  dipping  maneuver  oaloulation 
and  display. 

5* 

COURSE 

Manually  enters  or  overrides  a  course  for  a  specified 
traok. 

6* 

A/C  SENSOR 

Enters  or  ohanges  any  sensor  status  for  eaoh  sensor 
of  an  ownship-oontrolled  ASW  A/C. 

7 

NON-REAL  TIME 

Enters  a  new  firm  non-real- time  unknown  track 
into  the  system. 

8« 

COMPUTE  HELO 
HEADING 

Requests  the  calculation  of  a  heading  and  speed 
for  a  desired  relative  wind  direction  and  speed 
across  the  deck  for  a  helo  launch  or  reoovery. 

9* 

RCMD  HEADING 

Indicates  to  the  Bridge  that  a  recommended  ownship 
heading  and  speed  has  been  calculated  for  helo 
launoh  or  recovery. 

10« 

TRACK  HISTORY 

Displays  or  deletes  the  display  or  traok  history 
points  fo»*  a  specified  traok. 

11* 

SPEED 

Manually  enters  or  overrides  a  speed  for  a  specified 
track. 

12 

DOWNED 

AIRCRAFT 

Enters  a  downed  A/C  symbol  at  the  ball  tab  co¬ 
ordinates. 

*  Function  required  in  the  training  soenario. 
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TABLE  E-7.  ASAC  ALTERNATE  ARRAY  FUNCTIONS,  CONT. 


VAB 

Label 

Function  Summary 

13 

COMP  LOFAR  FIX 

Displays  or  deletes  a  comparative  LOFAR  Fix. 

14 

SONOBUOY/TIME 

Enters  a  sonobuoy  special  point  and  associated 
data  of  type,  time  remaining,  and  holding  or  not 
holding  contact. 

1 5* 

ENDRNCE 

Enters  or  changes  the  enduranoe  time  for  an  own- 
ship  controlled  ASW  A/C. 

16 

WEAPON  TYPE 

Enters,  updates,  or  displays  the  number  of  weapons 
on  board  an  ownship-oontrolled  ASW  A/C. 

17 

SONOBUOY  RF 

CHN 

Enters  the  Radio  Frequency  (RF)  ohannel  for  a 
specified  sonobuoy. 

18 

TRANSDUCER 

Enters  the  transducer  depth  of  a  specified  sonobuoy. 

•  Function  required  in  the  training  scenario 
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TABLE  E-8.  ASW  POINTS  ARRAY  FUNCTIONS. 


VAB 

Label 

Function  Summary 

1* 

REF  POINT 

Enters  a  reference  point  fixed  speoial  point  at 
the  ball  tab  coordinates. 

2 

2 

Not  implemented. 

3 

3 

Not  implemented. 

4 

4 

Not  implemented. 

5* 

COURSE 

Enters  a  course  value,  specified  in  DDEUS  3»  4, 
and  5»  for  a  special  point  or  ownshlp. 

6 

DRAW  LINES 

Draws  utility  lines  on  the  PPI.  Two  actions  are 
required  to  draw  a  line,  one  for  the  beginning  point 
and  one  for  the  ending  point. 

7 

ACOUS  BNG 

Enters  a  non-real- time  acoustic  bearing  on  a  friend 
surface  or  subsurfaoe  trade.  Helo  or  sonobuoy  in 
close  contact. 

8 

ACOUS  FIX 

Enters  an  aooustic  fix  special  point  at  the  ball 
tab  coordinates. 

9 

SURFACE  ZERO 

Enters  a  surface  zero  special  point  at  the  ball 
tab  coordinates. 

10* 

RADAR 

SINKER 

Enters  a  radar  sinker  special  point  at  the  ball 
tab  coordinates. 

11« 

SPEED 

Enters  a  speed  value,  specified  in  DDEUs  4  and  5> 
for  a  special  point  or  ownship. 

12* 

REPOS 

Enters  a  new  position  for  ownship  or  a  special  point 
in  close  oontrol.  The  new  position  is  specified 
by  the  ball  tab  position. 

*  Function  required  in  the  training  soenario. 
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TABLE  E-8.  ASW  POINTS  ARRAY  FUNCTIONS,  CONT. 


VAB 

Label 

Function  Sums  ary 

13* 

MAD  CONTACT 

Enters  a  MAD  oontaot  fixed  special  point  at  the 
ball  tab  coordinates. 

14* 

DATUM 

Enters  a  Dat-m,  changes  a  subsurface  track  to  a 
Datum  or  enters,  updates,  or  deletes  alphanumeric 
designation  of  a  specified  Datum. 

15 

BRIEF 

CONTACT 

Enters  a  brief  oontact  special  point  at  the  ball 
tab  coordinates. 

16* 

ASM  SEARCH 
CENTER 

Enters  an  ASW  search  oenter  special  point  at  the 
ball  tab  coordinates. 

17 

ALL  CNSLS 

Displays  a  reference  point  or  line  speoial  point 
at  all  oonsoles. 

18* 

TORPEDO 

Enters  a  torpedo  special  point  at  the  ball  tab  co¬ 
ordinates. 

•  Function  required  in  the  training  scenario. 
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ASWFCO  CONSOLE  FUNCTIONS 

The  ASWFCO1 s  mode  selection  capability  is  shown  as  an  array  tree  in 
Figure  £-9.  Of  the  possibilities  shown,  the  training  system  scenario  requires 
that  the  ASWFCO  use  the  ASWFCO  primary  array  (shown  simply  as  "ASWFCO'4  in 
the  figure),  the  ASWFCO  alternate  array  (ALTN  ASWFCO),  and  the  ASW  data  array 
(ASW  DATA). 

The  ASWFCO  primary  array  is  shewn  in  Figure  E-10.  It  allows  the  ASWFCO 
to  oontrol  the  ship's  underwater  weapons  systems,  and  to  respond  to  orders 
from  the  ASWOC.  Using  the  functions  in  this  array,  the  ASWFCO  oan  put  in 
weapons  offset,  oan  order  a  manual  over  the  side  (OTS)  attack,  and  display 
the  attack  lines.  A  brief  description  of  these  functions  is  provided  in 
Table  E-9. 

The  ASWFCO  alternate  array,  shown  in  Figure  E— 11,  allows  the  ASWFCO 
to  gather  or  modify  data  about  the  target  in  order  to  make  better  recommendations 
to  the  ASWOC  and  to  sonar.  Using  this  array,  trial  maneuvers  oan  be  aeleoted, 
target  depth  oan  be  determined,  and  the  target  symbol  oan  be  repositioned. 
These  functions  are  summarized  in  Table  E-10. 

The  ASW  data  entry  array,  shown  previous.  /  in  Figure  E-4  and  described 
in  Table  E-4,  is  used  by  the  ASWFCO  in  evaluating,  deteoting,  and  tracking 
targets.  It  also  provides  ownship  weapon  and  sensor  data. 


MOOE 

SELECT 


NAVTFAEQUIPCEN  80-C-00  95-1 


HOLD 

FIRE 

DISPLAY 

ASROC 

DATA 

RCMD 

HEADING 

• 

WEP 

OFFSET 

• 

DES1G 

SONAR 

UNIT* 

USE 

PRED 

TRACK 

0^ 

© 

@ 

TARGET 

KILLED 

FIRED 

ASROC 

DATA 

DISPLAY 

ATTACK 

EVAL 

DISPLAY 

ATTACK 

LINES 

MANUAL 

OTS 

ATTACK* 

© 

© 

© 

*  indicate* «  DDEU  entry  accompanies  the  VAB  action. 


Figure  E-10.  ASWFCO  pr inary  array, 
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TABLE  E-9.  ASWFCO  PRIMARY  ARRAY  FUNCTIONS. 


VAB 

Label 

Funotion  Summary 

1 

CMaa/Tii 

Indioates  one  of  three  specific  values  of  thrust 
cutoff  velocity  (DMma)  and  airframe  separation  time 
(Til)  to  be  'sent  to  the  MSP  for  use  in  a  UFCS  system 
operability  test. 

2 

2 

Not  implemented. 

3 

DELETE  C 

O.'FSET 

Deletes  anj  offset  in  effect.  If  at  OSC  console 
FIXED  OFFS. ST  must  be  selected. 

U 

GEO  OFFSET 

Causes  the  PPI  symbology  at  the  requesting  console 
to  be  displayed  relative  to  a  specified,  fixed  geographic 
point  and  places  the  console  in  an  offset  mode. 
If  at  OSC  console  FIXED  OFFSET  must  be  selected. 

5 

ENTiiR  U?  WPN 
AVAIL 

Enters  weapon  availability  status  of  the  magazines 
for  ASROC  torpedo,  AS  ROC  de  i  charge,  and  OTS  torpedoes, 
and  causes  the  weapon  availability  status  to  be 
displayed  at  the  WCP. 

6* 

CANTHO 

Notifies  ASWOC  (or  SWC  if  ASWOC  is  off-line)  that 
ASWFCO  cannot  ocmply  with  an  assignment  or  an  engagement 
order,  in  response  to  an  assign  or  engage  alert. 

7* 

HOLD  FIRE 

Alerts  ASWFCO  to  hold  fire  on  the  track  in  close 
oontrol. 

8* 

DISPLAY 

AS  ROC  DATA 

Displays  ordered  and  observed  launcher  train  and 
elevation. 

9* 

RCMD  HEADING 

Forwards  a  Mk  116  computed  or  manually  entered  recom¬ 
mended  heading  line  and  alert  to  the  Bridge  console 
when  required  to  unmask  the  ASROC  launcher,  bring 
a  target  within  weapon  range,  or  obtain  a  launch 
position  for  OTS  torpedoes. 

•  Funotion  required  in  the  ^raining  soenario. 
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TABLE  E-9.  ASWFCO  PRIMARY  ARRAY  FUNCTIONS,  CONT. 


VAB 

Label 

Function  Summary 

10* 

WEP  OFFSET 

Modifies  the  aimpoint  position  for  ASROC  and  OTS 
torpedoes  or  ASV  aircraft  weapons  to  the  position 
specified.  Also  deletes  an  existing  offset  for 
a  specified  WEP  and  causes  recomputation  of  WEP 
with  no  offset. 

11* 

DESIG  SONAR 

UNIT 

Orders  a  sonar  unit  to  search  a  designated  position 
for  a  contact. 

12* 

USE  PRED 

TRACK 

Orders  the  use  of,  or  termination  of  the  use  of, 
predicted  track  data  for  the  sonar. 

13 

* 

TARGET  KILLED 

Notifies  SWC,  Bridge,  ASWOC  and  other  PUs  that  a 
target  has  been  destroyed  by  ownahip  ASW  weapons. 

14* 

FIRED  ASROu 
DATA 

Causes  a  display  of  ASROC  firing  data  that  was  saved 
at  the  time  of  the  most  reoent  weapon  away  signal. 

15 

DISPLAY 

ATTACK  EVAL 

Requests  the  DRO  display  of  ASROC  attaok  evaluation 
data  at  time  of  interoept  or  OTS  attack  evaluation 
data  at  the  time  of  torpedo  launch. 

16* 

DISPLAY 

ATTACK  LINES 

Causes  the  PPI  display  of  calculated  attack  geometry 
for  an  ASROC  or  OTS  torpedo  engagement  when  an  attaok 
solution  exists. 

17 

DESIG/ DELETE 
ASWFCO 

Addresses  the  symbol  in  close  control  to  ASWFCO 
and  places  an  alpha  symbol  on  the  track  designating 
the  sensor  sr.uroe  or  deletes  the  specified  symbol 
and  alpha  designation  fr^ui  ASWFCO  console  address 
list. 

18* 

MANUAL  OTS 
ATTACK 

Initiates  or  terminates  the  engagement  of  a  target 
using  an  OTS  torpedo  when  the  WCP  is  not  available. 

*  Function  required  in  the  training  scenario. 
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Figure  E-11.  Alternate  ASWFCO  array. 
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TABLE  E-10.  ASWFCO  ALTERNATE  ARRAY  FUNCTIONS. 


VAB  Label  Function  Summary 


1  DISPLAY  Orders  the  display  of  data  specified  by  suooeeding 

VAB  actions  of  WIND  SPEED  DIRECTN,  PORT  ANEM,  or 
STBD  AN EH  VAB. 

2  2  Not  implemented. 

3*  ASSOC/DISASSOC  Associates  or  disassociates  two  specified  OSS  tracks. 

H  MANUAL  TMA  Initiates  manual  TMA  override  for  an  OSS  track 

or  terminates  manual  TMA  override  for  an  OSS  track 

or  PK  track. 

5*  COURSE  Enters  a  manual  value  for  oourse  on  the  specified 

UFCS-reported  (OSS  or  Mk  116  manual  subsilrfaoe) 
track  or  ownship  when  specified. 

6*  NEW  TRACK  Manually  enters  a  Mk  116  manual  subsurface  track 

at  the  specified  coordinates. 

7  PORT  ANEM  Enables  port  anemometer  inputs  for  system  use, 

provides  display  of  ourrent  wind  data  from  the 
port  anemometer,  and  disables  inputs  from  the  starboard 
anemometer.  When  this  VAB  action  follows  a  DISPLAY 
VAB  action,  wind  data  from  port  anemometer  inputs 
is  displayed  in  the  Wind  Data  DRO. 

8  STBD  ANEM  Same  as  described  above  for  PORT  ANEM,  except  applied 

to  starboard  anemometer. 

9  WIND  SPEED  Manually  enters  and  displays  a  value  for  wind  speed 

DIRCTN  and  direction  or  when  this  VAB  action  follows  a 

DISPLAY  VAB  action,  causes  the  display  of  the  Wind 
Data  DRO  containing  the  current  wind  speed  and 
direction  in  use  by  the  system. 
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TABLE  E-10.  ASWFCO  ALTERNATE  ARRAY  FUNCTIONS,  CONT. 


VAB 

Label 

Function  Summary 

11* 

SPEED 

Enters  a  manual  value  for  speed  on  the  specified 
UFCS-reported  {OSS  or  Mk  116  manual  subsurface) 
track  or  ownship. 

12* 

REPOS 

Repositions  the  specified  UFCS-reported  (OSS  or 
Mk  116  manual  subsurface)  traok  to  the  position 
of  the  ball  tab. 

13 

13 

Not  implemented. 

14* 

TRIAL  MANVR 

Requests  the  calculation  and  display  of  a  collision 
intercept  between  a  track  and  another  traok,  special 
point,  or  designated  looation. 

15* 

TRACK  DEPTH 

Enters  a  manual  depth  or  uses  the  doctrine  value 
for  depth  on  the  specified  UFCS-reported  (OSS  or 
Mk  116  manual  subsurface)  track. 

16 

16 

Not  Implemented. 

17* 

TRACK  MNVfiNG 

Notifies  the  sonar  supervisor  via  the  Sonar  Remote 
DRO  that  an  apparent  track  maneuver  has  been  observed 
by  the  ASWFCO. 

18 

18 

Not  implemented. 

*  Function  required  in  the  training  scenario. 
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WEAPONS  CONTROL  PANEL 

In  addition  to  NTDS  console  functions,  the  ASWFCO  also  makes  use  of 
the  Weapons  Control  Panel  (WCP).  This  is  a  sophisticated  system  that  gives 
the  ASWFCO  oontrol  over  the  weapons  system,  provides  up-to-date  information 
about  the  status  of  the  weapon  system  and  ship's  sensors,  and  also  provides 
computer  recommendations  for  heading  and  firing  solutions.  Figure  E-12  shows 
the  WCP.  The  buttons  and  indicators  on  the  panel  have  been  grouped  visually 
by  means  of  dashed  lines  into  functional  units  for  the  sake  of  discussion. 
The  shaded  buttons  and  indicators  in  this  and  subsequent  figures  are  not 
needed  in  the  training  soenario. 

UPPER  CONTROL  PANEL  CONTROLS  AND  INDICATORS.  Figure  E-1*!  shows  the  upper 
oontrol  panel  controls  and  indicators  in  more  detail.  Tho  only  oontrols 
needed  in  this  portion  of  the  WCP  are  the  power  and  intensity  oontrols. 
Illumination  of  these  buttons  is  also  shown. 

SYSTEM  MODE  SELECTION  CONTROLS  AND  INDICATORS.  Figure  E-14  shows  the  system 
mode  selection  controls  and  indicators.  The  oontrols  are  not  needed  in  the 
training  scenario,  but  the  normal  illumination  of  the  indicators  does  convey 
information  and  is  shown. 

SYSTEM  STATUS  INDICATORS.  This  two-part  indicator  is  illustrated  in  Figure 
E— 15.  Table  E-11  shows  the  subset  of  the  displays  needed  in  the  training 
scenario. 

SYSTEM  INTERLOCK  S1ATUS  INDICATORS.  This  six-part  indicator  is  illustrated 
in  Figure  E-16.  Table  E-12  describes  the  subset  of  displays  needed  in  the 
training  soenario. 

FIRE  SWITCH  AND  FIRING  SEQUENCE  INDICATORS.  The  fire  switch  and  firing  sequence 
indicators  are  illustrated  in  Figure  E— 17 -  The  elements  are  described  in 
Table  E-13 . 

WEAPON  INVENTORY  ASSIGNMENT  AND  TRIAL  SOLUTION  CONTROLS  AND  INDICATORS. 
These  controls  and  indicators  are  illustrated  in  Figure  E-18.  The  switches 
cause  the  displays  to  indicate  what  is  in  each  cell  or  tube.  The  details 
are  given  in  Table  E-14. 

TMA  SOURCE,  SENSOR  STATUS,  AND  SYSTEM  CLEAR  CONTROLS  AND  INDICATORS.  These 
controls  and  indicators  are  illustrated  in  Figure  E-»  1 9 .  Their  functions 
are  described  in  Table  E~15. 

CELL  STATUS  AND  SELECTION  CONTROLS  AND  INDICATORS.  The  oell  status  oontrols 
are  shown  in  Figure  E-20.  Their  function  is  desoribed  in  Table  E-16.  In 
the  training  soenario,  only  one  call  is  needed.  When  the  target  is  within 
a  certain  range  and  the  oomputer  recommends  an  ASROC,  cell  1  should  have 
a  yellow  bar  displayed.  When  it  is  selected,  the  green  flood  should  be  dis¬ 
played.  These  indicators  are  off  if  a  Mk  46  OTS  torpedo  is  selected. 
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Figure  E-12.  Weapons  Control  Panel 
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Figure  E-15.  System  Status  indicators. 

TABLE  E-11.  SYSTEM  STATUS  INDICATOR  DISPLAYS. 


Indi-  Display/Color  Meaning 

oator 


Top 

Approved  Torpedo 

Not  Seleoted 

Displayed  when  WCP  status  input  indicates  torpedo 
approved  interlook  is  satisfied  and  the  Permission 
To  Fire  interlock  is  not. 

Approved  Depth 
Charge  Not 

Seleoted 

Displayed  when  WCP  status  input  indicates  depth 
charge  approved  interlock  is  satisfied  and  the 
Permission  To  Fire  interlock  is  not. 

Bulkhead-Mounted 
Control  Box  (BMCB) 
in  Local/Red  flood 

Displayed  when  TSP  is  available  and  torpedo  launching 
system  is  not  in  Remote. 

System  Ready/ 

Green  flood 

Displayed  when  no  fault  or  failure  legend  is 
illuminated  in  either  the  top  or  bottom  System 
Status  Indicator. 

Red  flood 

Displayed  as  shown  above,  or  if  no  alphanumeric 
legend  is  ordered  and  a  red  flood  is  displayed 
on  the  bottom  System  Status  indicator. 

Bottom 

Select  Torpedo 

Not  Approved 

Displayed  when  WCP  status  indicates  the  Torpedo 
Selected  interlook  in  the  WCP  is  actuated  and 
Permission  To  Fire  interlock  is  not. 

iiiiiuiiin.-HMim^i^mpppi.ppp.i  mi  u.i.m 
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TABLE  E-11.  SYSTEM  STATUS  INDICATOR  DISPLAYS,  CONT. 


Indi¬ 

cator 

Diaplay/Color 

Meaning 

Seleot  DC  Not 
Approved 

Displayed  when  WCP  status  inputs  indicate  the 
Depth  Charge  Selected  interlock  in  the  WCP  is 

actuated  and  the  Permission  To  Fire  interlook 
is  not. 

Bottom 

Red  flood 

Displayed  when  no  legend  is  ordered  and  Red  flood 
is  displayed  on  top  System  Status  indioator  or 
in  conjunction  with  Fault,  Mode  Error,  of  DASCO 
Error  indicators  if  ICSS  Error  is  not  displayed. 

Green  flood 

Displayed  in  conjunction  with  Ordered  Needing 
Matohed,  or  if  no  legend  is  ordered  and  Green 
flood  is  displayed  on  top  System  Status  indicator. 
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Figure  E-16.  System  Interlock  Status  Indicators. 


TABLE  E-12.  DESCRIPTION  OF  SISTEM  INTERLOCK  STATUS  INDICATORS. 


Indicator  Color 


Description 


TARGET  IN  Off 
RANGE 

Green 


Not  used. 

When  an  ASROC  torpedo,  ASROC  depth  charge,  or  OTS 
torpedo  is  assigned  or  an  ASROC  or  OTS  torpedo  is 
recommended  with  no  weapon  assigned  and  the  oaloulated 
WEP  is  within  the  effective  range  of  the  specified 
weapon. 

When  off  or  green  state  is  not  applicable.  There 
is  not  necessarily  an  opening  of  the  firing  circuit 
when  this  indicator  is  r6d,  but  all  other  interlocks 
being  satisfied,  a  weapon  launched  under  these  circum¬ 
stances  has  a  very  low  kill  probability. 
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TABLE  B-12.  DESCRIPTION  OF  SYSTEM  INTERLOCK  STATUS  INDICATORS,  CONT. 


Indicator  Color  Description 
Off  Not  used. 

Green  When  a  TMA  source  is  selected  and  an  OTS  torpedo  is 

assigned,  or  an  ASROC  torpedo  or  ASROC  depth  oharge 
is  assigned  and  ooaputed  WEP  range  is  greater  than 
the  specified  minimum.  (The  ooaputer  controlled  interlock 
in  the  WCP  will  be  ordered  actuated  in  the  Normal 
ASW  and  Training  With  Launoher  modes  when  an  ASROC 
weapon  is  assigned  and  the  Solution  Safe  indioator 
is  green.) 

Red  When  off  or  green  is  not  applicable. 

Off  Not  used. 

Green  OTS  torpedo  is  assigned  and  the  Sector  Clear  interlook 
on  the  TSP  is  satisfied;  or  ASROC  torpedo  or  ASROC 
depth  oharge  is  assigned  or  recommended  and  a  cell 
is  not  selected  and  the  computed  value  of  launcher 
train  is  within  the  stored  clear  firing  arc;  or  ASROC 
torpedo  or  ASROC  depth  oharge  is  assigned  and  a  oell 
selected  and  the  Launcher  Sector  Clear  interlock  is 
satisfied. 

Red  When  off  or  green  is  not  applicable. 

WEAPON  Off  Not  used. 

READY 

Green  When  an  ASROC  torpedo,  ASROC  depth  oharge,  or  OTS 
Flashing  torpedo  is  assigned  and  the  Weapon  Ready  interlock 
in  the  MSP  or  TSP  is  satisfied. 

LAUNCHER  Green  Launcher  ready  indicator. 

READY 

PERMISSION  Off  Not  used. 

TO  FIRE 

Green  ASROC:  When  an  ASROC  46  or  ASROC  DC  is  assigned, 
the  status  input  from  the  Weapon  Status  Approval  Panel 
indicates  that  the  appropriate  weapon  is  approved, 
the  Permission  to  Fire  signal  is  received  from  the 
WCP,  and  the  Weapon  Select  switch  is  in  the  Missile 
position. 


SECTOR 

CLEAR 


SOLUTION 

SAFE 
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TABLE  B-12.  DESCRIPTION  OF  SYSTEM  INTERLOCK  STATUS  INDICATORS,  CONT. 


Indicator  Color  Description 


PERMISSION  Green,  OTS:  When  an  OTS  is  assigned  at  the  WCP,  the  Permission 
TO  FIRE,  cont.  to  Fire  signal  is  received  from  the  WCP,  and  the  Weapon 
oont.  Select  svitoh  is  in  the  Mk  32  TT  position. 


Red 


When  off  or  green  is  not  applicable 


Switoh/ 

Indicator 
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TABLE  E-13.  FIRE  SWITCH  AND  FIRING  SEQUFNCE  INDICATOR  FUNCTIONS 


/  Color/  Description 


Color/ 

Display 


Firing 

Switoh 

N/A 

The  filing  switoh  mur*  pushed  in,  then  turned. 

It  must  be  held  or.  it  will  automatically  return 
to  the  center  posits. ».  It  must  be  turned  to  Standby 
before  Fire. 

Standby 

Green 

Illuminated  when  the  Firing  Switoh  if  turned  to  Standby, 

Fire 

Qreen 

Illuminated  when  the  Firing  M„itch  is  turned  to  Fire. 

Firing 

Sequence 

FIRING 

SKQUflTB 

Displayed  when  a  Tiring  sequence  has  not  been  initiated 
or  after  a  firing  sequence  when  the  status  is  cleared 
by  a  change  iri  the  Cell/Tube  -jleot  order. 

ST/'JDBY 

Displayed  when  WCP  status  input  indicates  that  Standby 
has  bean  ordered  and  the  Ready  To  Fire  condition  does 
not  exiut. 

READY  TO 

FIRE/ 

Green 

Disrlayad  when  Ready  To  Fir?  condition  exists  (3  seoonds 
after  Standby  for  OTS  or  MSP  input)  and  Fire  order 
has  not  been  iusued. 

FIRE  ORD- 
ERSD/Red 

Displayed  when  Fire  has  been  ordered  and  System  Misfire 
has  not  been  preao' t  for  1  aeoond,  r :  has  been  superseded. 

WEAPON 

AWAY/ 

Green 

Displayed  oosequent  to  weapon  launob.  This  condition 
will  cause  the  inventory  for  the  seleoted  oell/tube 
to  be  changed  to  empty  immediately. 
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Figure  E-1 8.  Weapon  Inventory  Assignment  and  Trial  Solution 
controls  and  indicators. 


TABLE  E-1 4,  DESCRIPTION  OF  THE  WEAPON  INVENTORY  ASSIGNMENT 
AND  TRIAL  SOLUTION  CONTROLS  AND  INDICATORS. 

Display/  Description 

Control 

ASROC  DC  During  Trial  Solution,  the  yellow  bar  displayed  when  an ’ASROC 

LAUNCHER  is  reoommended  by  the  computer  and  the  switch  is  not  depressed; 

AND  MAC-  green  flood  when  depressed. 

AZINE 

OTS  DC  (Port  and  starboard)  Green  flood  when  depressed. 

MAGAZINE 


TRIAL 

SOLUTION 


Works  in  conjunction  with  the  REQUEST  DIR  switch  on  the  TMA  Souroe 
Senror  Status  bank  of  switches.  When  depressed,  the  REQ  DIR 
acts  as  a  hold  trial  solution  switch.  Green  flood  when  depressed. 
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Figure  E-19*  TMA  Source,  Sensor  Status,  and  System  Clear 
controls  and  indicators. 


TABLE  E— 15.  FUNCTION  OF  TMA  SOURCE,  SENSOR  STATUS,  AND 
SYSTEM  CLEAR  CONTROLS  AND  INDICATORS. 


Number  Display/Color  Description 


1  SONAR  1  SEARCH  Displayed  when  Sonar  Channel  1  is  on-line,  not 

in  oont.vat,  and  no  position  kept  traok  originated 
from  Channel  1  input  is  in  Traok  Stores;  flashed 
for  the  first  3  minutes  after  the  sonar  operator 
switohes  from  contact  to  search. 


2 


SONAR  1  TMA  GOOD  Displayed  when  Sonar  Channel  1  is  in  oontact, 

the  quality  of  the  automatic  TMA  solution  is  good, 
and  the  predicted  track  is  not  in  use. 


PREDICTED  TRACK  Displayed  when  Sonar  Channel  1  is  in  oontact, 

GOOD  the  quality  of  the  automatic  TMA  solution  is 

good,  and  the  predicted  track  is  in  use, 


Green  flood  Displayed  when  SONAR  1  seleot  switch  is  depressed 

and  selection  is  legal. 

SONAR  2  SEARCH  Displayed  when  Sonar  Channel  2  is  on-line,  not 

in  oontact,  and  no  position  kept  traok  originated 
from  Chunriel  2  input  is  in  Track  Stores;  flashed 
for  the  first  3  minutes  after  the  sonar  operator 
switches  from  oontact  to  search. 


IS2 
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TABLE  E— 15*  FUNCTION  OF  TMA  SOURCE,  SENSOR  STATUS,  AND 
SYSTEM  CLEAR  CONTROLS  AND  INDICATORS,  CONT, 

Number  Display/Color  Description 

2,  SONAR  2  TMA  GOOD  Displayed  when  Sonar  Channel  2  is  in  contact, 
oont.  the  quality  of  the  automatic  TMA  solution  is 

good,  and  predicted  track  is  not  in  use. 

PREDICTED  TRACK  Displayed  when  Sonar  Channel  2  is  in  contact, 

GOOD  the  quality  of  the  automatic  TMA  solution  is 

good,  and  predicted  track  is  in  use. 

Green  flood  Displayed  when  SONAR  2  select  switch  is  depressed 

and  selection  is  legal. 

3  SYSTEM  CLEAR  When  depressed,  all  WCP  orders  to  the  operational 

program  are  oanoeled  exoept  System  Mode  order 
and  Trial  Solution  Hold  (if  implemented).  A 
Trial  Solution  Hold  order  is  oanoeled  when  SYSTEM 
CLEAR  is  depressed  a  second  time.  Break  engage 
processing  is  performed  for  the  target  identified 
by  the  selected  data  source.  Associated  attack 
geometry  is  deleted  from  the  UYA-h  oonsole. 
Computations  and  hardware  output  relating  to 
the  seleoted  weapon  system  are  stopped. 

4  CDSS/Green  flood  The  CDSS  select  switoh  orders  utilisation  of 

traok  input  data  from  the  CDSS  or  NTDL  for  generation 
of  an  attack  solution.  It  places  a  CDSS  surfaoe 
or  subsurface  traok  in  close  oontrol  at  the  ASWFCO 
oonsole.  The  green  flood  is  displayed  when  the 
switoh  is  depressed  and  the  solution  is  a  legal 
action. 

CDSS  RADAR  iARGET  Displayed  when  the  source  of  an  NTDd-reported 

subsurface  track  is  identified  ac  a  relar. 

CDSS  REMOTE  TRACK  Displayed  when  an  N 'IDS- re  ported  subsurface  traok 

is  not  Identified  us  a  radar  traok  (e.s»»  remote 
traok,  CDSS  position  kept  track,  etc.). 

This  oontrol  is  used  to  place  a  manual  anbsnrfaoe 
track  in  close  oontrol  at  the  ASWFCO  console, 
Green  flood  is  displayed  when  this  switch  i.* 
selected  legally. 


5 


MANUAL  TRK/Green 
flood 
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TABLE  E-15.  FUNCTION  OF  TMA  SOURCE,  SENSOR  STATUS,  AND 
SYSTEM  CLEAR  CONTROLS  AND  INDICATORS,  CONT. 


Number 

Display/Color 

Description 

5, 
oo  nt. 

MANUAL  TRACK 

ENTERED 

Displayed  when  a  aanual  subsurface  traok  is  under 
assignment  or  engagement  from  the  WCP  or  when 
a  manual  subsurface  traok  symbol  is  in  dose 
control  and  the  track  is  not  under  engagement 
from  the  WCP. 

6 

REQUEST  DIR 

Displayed  wnen  Trial  Solution  Hold  is  not  enabled 
and  Request  Director  Select  switch  is  not  depressed, 
or  after  depressing,  a  response  to  request  has 
not  been  reoeivod  from  CDSS.  The  switch  causes 
a  message  to  be  sent  to  NTDS  giving  coordinates 
of  the  ASROC  WEP  and  a  TRACK  WEP  REQ  alert  is 
sent  to  SWC. 

Yellow  bar 

Displayed  when  REQUEST  MR  aoleot  switoh  has 
not  bean  decreased,  an  ASROC  torpedo  or  ASROC 
depth  oharge  is  assigned,  and  all  Interlook  Status 
indicators  are  green. 

Red  flood 

Illegal  REOUEST  DIR. 

6 

Greers  bar 

Displayed  when  REQUEST  DIR  hes  been  depressed 
and  an  assigned  message  is  not  received  from 
CDSS. 

DIRECTOR  ASSIGNED/ 
Green  flood 

Displayed  when  REQUEST  DIR  has  been  depressed 
and  an  assigned  message  is  received  from  CDSS. 

TRIAL  SOLUTION 

HOLD 

This  legend  replaces  REQUEST  DIR  legend  when 
TRIAL  SOLUTION  Is  depressed.  It  is  displayed 
when  trial  solution  is  ordered  *t  WCP  until  solution 
hold  is  uleared,  or  if  hold  Is  not  ordered,  until 
trial  dolution  is  cleared. 

Green  flood 

Displayed  when  Trial  Solution  Hold  aeleot  switch 
is  depressed  until  Hold  condition  is  cleared. 

m 


Figure  E-20.  Cell  Status  and  Solectron  Controls  and  Indicators. 


TABLE  E-16.  FUNCTION  OF  CELL  STATUS  AND  SELECTION  CONTROLS 
AND  INDICATORS. 


Display/ 

Color 

CELL  7 

AfiROC 
Mk  46 

CELL  # 

ASROf; 

DC 

Croon  flood*'' 


ielluv*  Oar1* 


Description 


Displayed  whan  the  inventory  for  oell  #  is  an  ASRGC 

Mk  45  torpedo.  In  the  training  roenario,  this  will  include 

cells  1,2,3|5»6,7,  as  shown. 

Displayed  when  the  inventory  for  oell  #  is  an  ASROU  depth 
charge.  In  the  training  scenario,  this  will  include 
culls  4  and  fi,  as  shot;n. 

Displayed  when  CELL  #  select  switch  is  depressed,  inventory 
for  oell  jiatohes  the  assigned  vreapon,  and  Launoher  Selector 
switches  have  responded  oorreotly, 

Displayed  when  CELL  t  select  switch  has  not  been  depressed 
and  oell  is  recommended  or  oell  was  selected  prior  to  TRIAL 
SOiN  HOLD  selected. 


^v’ptst  These  are  off  if  a  Mk  46  OTS  torpedo  is  selected. 
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*  > 

TUBE  STATUS  AND  SELECTION,  CELL  AND  TUBE  CLEAR  CONTROLS  AND  INDICATORS. 

Tub®  status  oontrols  are  illustrated  iu  Figure  E-ST.  They  are  deaoribed 
in  Table  E— 1 7 •  These  Indicators  are  off  if  an  ASROC  has  been  selected. 

In  the  training  soenario,  only  one  port  ai.'d  one  starboard  tube  is  needed. 

When  an  OTS  torpedo  is  seleoted  at  the  weapon  assignment,  a  yellow  bsr  should 
appear  in  tube  1  and  2. 

SEARCH  DEPTH  AND  GTRC  ANQLE/SEARCH  CONTROLS  AND  INDICATORS.  These  oontrols 
and  indicators  are  shown  in  Figure  E-22  and  are  deaoribed  in  Table  E-18. 

Sinoe  in  the  training  soenario,  the  ASROC  depth  charge  must  not  be  seleoted, 
and  sinoe  the  depth*  are  predetermined,  a  depth  should  be  seleoted  and  the 
yellow  bar  should  be  used  to  recommend  it.  In  this  way,  only  the  unshaded 
switch  in  the  upper  bank  needs  to  be  functional.  A  similar  approaoh  can 
be  taken  in  the  lower  bank,  and  again  only  the  one  unshaded  switch  would 
need  to  be  functional. 
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CELL* 

TUBE 

CLEAR 
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Figure  E-Jsl.  Tube  Status/Seleotion,  Cell  and  Tube  Clear 
controls  and  indicators. 


TABLE  E-17,  DESCRIPTION  OF  TUBE  STATUS/ SELECT ION,' CELL  AND  TUBE  CLEAR 
CONTROLS  AND  INDICATORS. 


Display/ 

Color 


Dosoript ion 


TUBE  «  PORT/ 
STARBOARD  MK  46 

Oreen  flood1** 


Yellow  bar* 


CELL  AND  TUBE 
CLEAR 


Displayed  when  the  inventory  for  the  tube  io  a  MK  46  torpedo. 


Dioplayed  whan  TUBE  #  select  witch  io  ceproosed,  inventory 
for  the  tvb*  matohec  the  assigned  weapon,  and  the  TSP  Tube 
select  uwitnli  has  responded  correctly. 

Displayed  vLun  '/USE  it  select  switch  han  cot  been  depressed 
and  the  tube  i#  rujoamended,  or  tube  was  selected  p"ior 
to  TRIAL  SOLN  HOLD  soleoted. 

Depressing  this  switch  cancels  any  previously  actuated 
CELL  or  TUBE  select  switches  or  Torpedo  setting  switcher 
and  their  resulting  actions  and  indications. 


•Note:  Thea^  are  off  if  an  ASROC  is  selected 


187 


i 


rrrsasammmstsmmm  r*A 


NAVTRAEQUIPCEN  80-C-0095-1 


TABLE  E— 18.  DESCRIPTION  OF  SEARCH  DEPTH  AND  GYRO  ANGLE/SEARCH 
CONTROLS  AND  INDICATORS  USED  IN  THE  SCENARIO. 


Display  Color  Description 

SEARCH  DEPTH  Displayed  at  all  times  during  Normal  ASW  mode 

unless  an  ASROC  depth  charge  is  assigned. 

Green  Displayed  when  a  SEARCH  DEPTH  #  seleot  switch 

flood  is  depressed  and  the  torpedo  mechanism  is  set 

at  Search  Depth  #.  This  is  equivalent  to  a  search 
depth  sync  indication. 

Yellow  Displayed  when  a  SEARCH  DEPTH  #  seleot  switch 

bar  has  not  been  depressed  and  Search  Depth  0  is 

recommended,  or  SEARCH  DEPTH  0  was  selected  prior 
to  the  TRIAL  SOLN  HOLD  selected. 

Displayed  when  no  weapon  is  assigned  or  OTS  Mk 
46  is  assigned  and  a  combination  of  Gyro  Angle 
0  and  Tube  Seleot  does  not  oause  torpedo  track 
to  oross  ownship  bow. 

Displayed  when  a  GYRO  ANGLE  0/SEARCH  MODE  #  select 
switch  is  depressed  and  the  torpedo  meohaniaia 
is  set  at  Gyro  Angle  0/Searoh  Mode  0.  This  is 
equivalent  to  a  Gyro  Angle/Searoh  Mode  sync  indi¬ 
cation. 

Yellow  Displayed  when  a  GYRO  ANGLE  0/SEARCH  MODE  seleot 

bar  switoh  has  not  been  depressed  and  Gyro  Angle 

0/Search  Mode  is  recommended,  or  GYRO  74  ANGLE 
0/SEARCH  MODE  #  was  seleoted  prior  to  TRIAL  SOLN 
HOLD  seleoted. 

SEARCH  MODE  it  Displayed  when  an  ASROC  torpedo  is  assigned. 
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APPENDIX  F 

COMMUNICATIONS  CIRCUITS 

The  members  of  the  CIC  team  use  a  variety  of  circuits  to  receive  and 
transmit  information.  The  internal  and  external  circuits  whiob  will  be  needed 
to  support  the  training  scenario  are  described  in  Tables  F-1  and  F-2  respec¬ 
tively. 


TABLE  F-1.  INTERNAL  COMMUNICATIONS  CIRCUITS. 


Circuit 

Type 

Stations 

Type  of  Information 

61JS 

Sound  powered 

ASWFCO, 

ASWOC, 

Sonar, 

North  Plotter, 
Bridge 

Flow  of  contaot  information; 
Shift  of  control 

29MC 

Multi-channel 

unit 

Sonar 

One  way  sonar  oontaot 
announcing 

1JS 

Sound  powered 

ASWOC*, 

Surface  Sup. , 
Bridge 

Ships  oontrol  information 
(e.g.  course,  speed  ohanges) 

21 JS 

Sound  powered 

Radar  oper, , 

Internal  CIC  information 

South  plotter, 
AS At*, 

ASWOC* 

(e.g,  range,  bearing  to  asaist 
ship  after  contacts) 

JC 

Sound  powered 

AS* FOC , 

Weapons 

Waaponr  status  tube  conditions 

Interphone 

NTDS  inter oom 
circuit 

ASWOC*, 

AS” FLO*, 

ALAC*, 

All  consoles* 

General  purpose  communications 
circuit  for  orders  and 
information 

BJF 

Sound  powered 

A3WFC0, 

Launoher 

Directions  to  the  launoh 
captain 

•Note*  At  the  stations  marked  with  an  asterisk,  a  light  alerts  the  station 
thdfc  another  station  wishes  to  communicate  over  the  circuit.  It  is  necessary 
to  switch  manually  to  the  circuit  to  oommunloate,  then  awitoh  baok  to  the 


normally  manned  circuit. 


r*s?  - 


NAVTRAEQUIPCEN  80-C-0095-1 

TABLE  F-2.  EXTERNAL  COMMUNICATIONS  CIRCUITS. 


Ciroult 

Type 

Stations 

Type  of  Information 

SAU 

Reporting 

Radio  telephone 

All  ships 

All  ships  of 
the  SAU, 
assist  ship, 
SAU  Commander, 
ASWOC 

All  oontaot/datum  information; 
maneuvers,  orders  for  SAU 

Submarine 

Coord. 

Radio  telephone 
Command  level 

SAU  Cmdr, 

OTC 

Information  about  oontact; 
taotios 

Air  Coord. 

Radio  telephone 
and  oontrol  units 

ACU,  Request  for  helos,  helo  state, 

SAU  Cmdr,  status,  RTB  notification 

Helo  home  plate, 

OTC 

ASW  Helo 
Control 

Radio  telephone 
helo  oontrol 

ASAC 

Helioopter 

Vectors,  MAD  reports,  air 
oontrol 

If  ' ' 


p,WlffWW"1Jr' 
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APPENDIX  G 

AUTOMATED  SPEECH  TECHNOLOGY  REQUIREMENTS 

The  Team  Training  system  will  require  the  use  of  both  the  automated 
speeoh  recognition  and  automated  speech  generation  technologies.  In  this 
appendix,  the  vooabulary  used  during  the  soenario  by  each  member  of  the  CIC 
team  and  by  supporting  team  members  is  defined  in  its  most  general  form. 
Although  it  is  not  within  the  scope  of  this  effort  to  do  detailed  trade-off 
analyses  of  the  presently  available  hardware  devices  and  software  techniques 
which  aould  support  training  system  needs,  the  vocabulary  lists  given  here 
will  provide  the  basis  for  conducting  such  studies.  A  concluding  section 
of  the  appendix  provides  checklists  for  evaluating  hardware  devices. 

SPEECH  RECOGNITION  REQUIREMENTS 

The  system  ooncept  described  in  this  report  involves  a  training  system 
for  three  members  of  the  CIC  t  am:  the  ASWOC,  ASWFCO,  and  ASAC.  Thus  the 
system  must  be  oapable  of  recognizing  and  understanding  the  vooabulary  used 
by  these  team  members.  The  distinction  between  "recognizing"  and  "understanding" 
is  crucial  here.  By  recognition  we  mean  the  ability  of  a  hardware  speech 
recognition  device  to  detect  that  a  particular  word  or  phrase  has  been  spoken. 
By  understanding,  we  mean  the  ability  of  the  training  system  as  a  whole  to 
respond  properly  to  an  input.  Only  the  speeoh  recognition  requirements  are 
described  here.  The  specification  of  system  responses  to  these  speeoh  inputs, 
the  understanding  function,  was  provided  in  the  classified  Appendix  C. 

The  phrase  lists  provided  here  have  been  arranged  to  show  complete  utterances 
rather  than  the  perticles  which  would  actually  be  recognized  by  the  hardware 
recognition  devioe.  The  maohine-dependent  task  of  defining  those  particles 
Is  beyond  the  soope  of  this  effort. 

A  feature  of  the  training  system  ooncept  is  its  capability  to  stand 
in  for  missing  team  members  by  providing  competent  team  member  models.  Thus 
the  vooabulary  lists  given  below  for  the  ASWOC,  ASWFCO,  and  ASAC  are  lists 
of  words  and  phrases  which  the  system  must  both  be  able  to  reoognize  and 
to  generate. 

Table  G-1  shows  the  special  symbols  used  to  define  the  vooabulary. 
Table  0-2  shows  the  words  used  by  all  team  members  on  all  cirouits.  Table 
G-3  provides  a  list  of  the  phrases  used  by  the  ASWOC,  arranged  by  circuit. 
This  arrangement  by  circuit  is  important  for  two  reasons:  Obviously  the 
speeoh  generation  system  needs  to  output  the  verbalizations  of  the  team  member 
models  over  the  appropriate  circuit.  More  importantly,  the  arrangement  provides 
a  powerful  method  of  partitioning  the  extensive  vocabulary  for  efficient 
opeech  recognition.  Tables  0-4  and  G-5  give  the  phrases  used  by  the  ASWFCO 
and  ASAC  respectively. 
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TABLE  G-1 .  MEANING  OF  SYMBOLS  USED  IN  THE  TABLES. 


Symbol  Meaning 


,  Pause  may  ocour. 

{}  Optional  word  or  phrase. 

/  Separates  mutually  exclusive  ohoioes  within  a  phrase. 

_  Used  to  remove  ambiguity  when  mutually  exclusive  ohoioes 

in  a  phrase  involve  more  than  one  word.  For  example,  the 
entry  "TDA/tango_delta_alpha"  means  that  either  the  abbreviation 
is  used,  or  the  three  phonetic  alphabet  elements. 

[]  Variable  information  of  the  type  specified  is  to  be  inserted, 

as  follows: 

[range]  and  [interval]  are  expressed  in  either  yardto  or 
miles: 

[digit]  {decimal  [digit]  {miles}}; 

[digit]  {answer}  {miles}; 

[whole  number]  hundred/thousand/answer  {yards}. 

[time]  is  expressed  as  follows: 

[digit]  [digit]; 

according  to  the  24  hour  clook. 

[speed]  is  expressed  in  knots: 

[digit]  {knots} ; 

[whole  number] /[digit  digit]  {knots}. 

[aourse],  [heading]  and  [bearing]  are  expressed  as  3  digits 
in  the  range  001  to  360.  (Course  can  also  be  given  as  000.) 

[alpha/bravo..,]  meant  a  word  from  the  phonetio  alphabet. 

[compass  point]  means  north/nortn  north  west/north  west/... 
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TABLE  02.  WORDS  AND  PHRASES  USED  BY  ALL  TEAM  MEMBERS. 


Circuit  Phrase 


All 


Roger 
Roger  out 
Over 
Out 
Aye 

Negative 

Tack 

Standby 

And 

Zero*,  one,  ...,  nine,  niner,  ten,  eleven,  t. ,,  ninety 
Alpha,  bravo,  ...,  Zulu 


•Note:  The  substitution  of  "oh"  for  "zero"  is  not  aooeptable  Navy  phraseology,, 
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TABLE  G-3.  PHRASES  USED  BY  THE  ASWOC. 

Circuit  Phrase 

SAD  Reporting  Interrogative  weapons  policy, 

Lima  2  on  station, 

Lima  2  laying  a  barrier  ?,xis  [true  compass  teearl&g]  Icing 
pin  [bearing]  degrees  [range]  Delta  Charlie/Liaia  2. 

Lime  2  {has}  MALcmn  [bearing]  Delta  CharlWLima  2  [range]. 

Conducting  HAD  vtjco  with  Lima  2, 

Lima  2  lost  contact  [bearing]  Delta  Charlie/Lima  2  [y'tuige], 
tracking  [oourae],  indioating/apeed  [speed] . 

Entering  TDA/tango_delta«,alphB. 

I  am  hot  [bearing]  {tack}  [range]. 

My  latest  [bearing]  {tack}  [range] . 

Classify  as  pos/prob/cert  sub  confidence  {level}  high/Iow, 
Execute  plan  red/blaek/ green/yellow. 

I  am  brother/aister. 

Coming  left/righfc  [course]. 

Prep  fcasoeit  port/starboard. 

Firing  bearing  [bearing], 

Bassett  away  bearing  [bearing]  time  [time], 

Bassett  splash,  water  entry  point  [bearing]  {t&ok}  [range]. 
Safe  time  [time]. 
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TABLE  G-3.  PHRASES  USED  BY  THE  ASWOC,  CONT. 

Circuit  Phrase 

6 US  Use  datum  as  search  center. 

Entering  TDA/tango_delta_alpha. 

{Lima  2}  MAEsnan  [hearing]  {tack}  [range]. 

Sonar  contact  [bearing]  {tack}  [range], 

v 

Classify  as  pos/prob/cert,  sub,  confidence  {level)  higL/low. 
We  are  the  attacking  ship. 

Weapons  policy:  weapons  free  pos  sub  confidence  level 
high  or  higher. 

Assign  MK  116  to  the  sonar  contact  bearing  [bearing]. 

Engage  MK  116. 

Prep  Bassett  port/starboard. 

Complete  your  firing  sequence. 

Shoot. 

Execute  to  follow  turn  [alpna/bravo, . .  ]  desig  [alpha/ bravo...] 
{tack}  [course]. 

When  executed  the  base  course  will  be  [oouroe]  zigzag  plan 
[alpha/bravo. . . ] , 

Execute  zigzag  plan. 

Plan  red/black/' green/yellow  exeouted. 

Come  left/' right  to  course  [course], 

Ner.t  course  will  be  [course]  at  time  [time]. 
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table  g phrases  used  by  the  aswoc,  cont. 

Circuit  Phrase 

61JS,  cent.  Weapons  policy:  weapons  free  poa  sub  confidence  level  high 

or  higher. 

Entering  TDA/tango._.delt».,_alpha. 

Execute  to  follow  turn  [alpha/bravo,..]  doslg  [alpha/bravo.,.] 
{tacit}  [course] . 

When  executed  the  base  course  will  be  [course]  zigzag  plan 
[alpbft/bravo...]. 

Execute  zigzag  plan, 

Come  left/right  to  course  [course]  tat  time  [time]}. 

Come  laft/right  steer  [course]. 

Next  course  will  be  [course]  at  time  [time], 

MADman  [bearing]  {taok)  [range]. 

Lima  2  lost  contact  [bearing]  {tack}  [range]. 

Firing  bearing  [bearing]. 

Bassett  away  [bearing]. 

Water  entry  point  [bearing]  {tack}  [range]. 
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TABLE  G-3«  PHRASES  DSED  BY  THE  ASWOC,  CONT. 

% 


Circuit  Phrase 


NTDS  Interphone  Lay  a  barrier  with  Lima  2  with  buoy  interval  of  [interval] 
axis  [true  compass  bearing]  from  [bearing]  {taok}  [range] 
from  datum. 

•  Weapons  polioy:  weapons  free  on  pos  sub  confidence  level 
high  or  higher,  Lima  2  weapons  free  after  third  MAD  oontact. 

Weapons  free  on  next  MAD  contaot. 

Sonar  contact  [bearing]  (tack)  [range]. 

Scram  Lima  2  [heading]/ [compass  point]. 

Prep  basaett  port/starboard. 


Bassett  away  port/starboard 
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TABLE  G-M.  PHRASES  USED  BY  THE  ASWFCO. 


Circuit  Phrase 

6 1JS  Sear oh  mode  ODT  PDT. 

The  helo  has  a  MAD  contact  [bearing]  {tack}  [range]. 

Shift  your  search  oenter  to  [relative  bearing] /datum. 

New  search  arcs  [relative  bearing]  to  [relative  bearing], 

Contaot  tracking  [course]  indicating/speed  [speed], 

Lima  2  lost  contact ,  last  known  position  [bearing]  {tack} 
[range]. 

Signal  strength  strong/weak,  signal  quality  sharp/fuzzy. 

Classify  as  pos/prob/oert  sub  confidence  {level}  high/ low. 

That  looks  like  Lima  2’s  MAD  contact  tracking  [course] 
Indicating/speed  [speed], 

MK  116  assigned. 

I  have/see  the  contaot  tracking  [course]  indicating/speed 
[speed] . 

I  am  offsetting  [range]  ahead/port_bow/starboard_bow  of 
the  contaot. 

Propping  bassett  port/starboard. 

I  have  a  green  board  one,  recommend  [ocurse]. 

Firing  bearing  [bearing]. 

Request  permission  to  fire. 

Green  board  two. 

Bassett  away  firing  bearing  [bearing], 

Bassett  splash,  water  entry  point  [bearing]  {taok}  [range]. 
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TABLE  G-4.  PHRASES  USED  BY  THE  ASWFCO,  CONT. 


Cirouit  Phrase 

61JS,  cont.  Listen  for  weapon  ntart  up. 

Shot  looks  good,  [range]  ahead/port_bow/starboard_bow  of 
contact. 


8JP  Matob  up  bearing  and  elevation  and  shift  to  remote. 

Insert  PI  plug. 
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TABLE  G-5.  PHRASES  USED  BY  THE  ASAC. 


Circuit  Phrase 

NTDS  Interphone  Lima  2  on  station  commencing  plant. 

Standby  for  Icing  pin. 

Hark  king  pin  {[bearing]  {tack}  [range]}. 

Plant  complete  axis  [true  compass  bearing]. 

MADman  MAEoan  [bearing]  {tack}  [range]. 

Conducting  MAD  vec  with  Lima  2. 

Lost  contact,  last  known  position  [bearing]  {tack}  [range]. 
Lima  2  clear  [compass  point]. 


ASW  Helo  Control  Lima  2  this  is  Freddy  override. 

Net 

Lima  2  lay  a  road  [true  compass  bearing]  from  position 
[bearing]  {tack}  [range]  Delta  Charlie/Lima  2,  interval 
[interval],  drop  on  my  command. 

Lima  2  vector  [heading]  for  station. 

Lima  2  port/starboard  [heading], 

Lima  2  standby  to  drop. 

Lima  2  drop  now. . .now. . .now. 

Lima  2  king  pin,. 

Lima  2  stand  by  {[range]}. 

Lima  2  execute  preview. 

Lima  2  vector  [heading]  for  ID/initial_drop_point. 

Lima  2  es  jcute  radar  MAD  vec. 

Lima  2  port  or  starboard  [1  or  2  digit(s)]  to  [1  or  2  digit(s)] 
us  needed. 


■tfifiWI 
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TABLE  G-5.  PHRASES  USED  BY  THE  ASAC,  CONT. 

Circuit  Phrase 

ASW  Helo  Control  Lima  2  mark  on  top. 

Net,  cont. 

Lima  2  port/ starboard  [heading]  Line_up_on_the_amoke/for_another 
_pass. 

Lima  2  weapons  free  classify  a  pos  sub  confidence  {level} 
high. 

Lima  2  I  am  hot  [bearing]  {tack}  [range]. 

Lima  2  scram  [heading]/[oompass  point]. 

Lima  2  prep  bassett  port/starboard. 

Lima  2  bassett  away  port/starboard. 
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SPEECH  GENERATION 

Speech  generation  is  often  regarded  as  a  much  more  simple  task  than 
speech  recognition.  In  a  sense,  this  is  true.  However,  the  extensive  speech 
generation  requirements  of  this  training  system  will  make  the  design  of  an 
adequate  speech  generation  system  a  challenge.  First  of  all,  it  must  provide 
distinguishable  speech  output  of  nine  different  talkers:  the  AJSWfC,  ASWFCO, 
ASAC,  plotter,  sonar  operator,  bridge,  SAU  commander  and  helicopter  pilot, 
and  instructor.  Furthermore,  it  should  be  capable  of  varying  the  output 
of  certain  messages  much  like  a  human  would.  As  a  simple  example,  it  should 
sometimes  say  "Aye"  and  at  other  times  "Roger."  It  also  must  choose  the 
appropriate  circuit  over  which  to  transmit  the  appropriate  voice  message. 

The  phrases  which  the  ASWOC,  ASWFCO,  and  ASAC  use  were  described  in 
the  preceeding  section.  In  the  following,  Tables  G-6  through  G-10  list  the 
phrases  used  by  the  plotter,  sonar  operator,  bridge,  SAU  oaanander  and  helioopter 
pilot  respectively.  In  addition,  all  models  should  have  the  capability  to 
speak  the  words  and  phrases  given  in  Table  G-1 . 

No  speeoh  messages  have  yet  been  defined  for  the  instructor  model, 
and  so  no  table  of  phrases  can  be  provided  at  this  time.  Nonetheless,  this 
model  should  be  taken  into  consideration  in  the  design  of  the  speech  generation 
capability,  with  provision  made  for  this  model  to  provide  set  up  descriptions 
and  extensive  feedback.  It  is  particularly  important  to  take  ease  of  modification 
into  consideration  in  the  design  of  the  instructor  model  speech  generation 
capability,  sinoe  one  of  the  purposes  of  the  system  is  to  investigate  the 
efficacy  of  various  forms  ol'  feedback. 
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TABLE  G-6.  PHRASES  USED  BY  THE  PLOTTER. 


Circuit 


Phrase 


Concur  ETA/echo_tango_alpha  to  TDA/tango_delta_alpha. 


Concur  cone  of  courses. 

Initial  course  for  zigzag  [course]. 


I  hold  us  entering  the  TDA/tango_delta*_alpha. 

I  see/hold  the  sub  tracking  [course]  indicating/speed  [speed]. 


TABLE  G-7 .  PHRASES  USED  BY  THE  SONAR  OPERATOR. 


6US  or  Tracking  the  contact  on  [alpha,  bravo. ..  ]  scan  [bearing] 

29MC  {tack}  [range]. 

Sonar  contact  [bearing]  {tack}  [range]. 

Signal  strength  strong/weak,  signal  quality  sharp/fuzzy. 

Classify  as  pos/prob/cert  sub  confidence  {level}  high/low. 
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TABLE  G-8.  PHRASES  USED  BY  THE  BRIDGE. 

Circuit  Phrase 

61 JS  Come  left/right  to  [course]. 

Next  course  will  be  [course]  at  time  [time]. 
Bearing  clear. 


TABLE  G-9.  PHRASES  USED  BY  THE  SAU  COMMANDER. 

Circuit  Phrase 

SAU  Reporting  Plan  red/black/green/yellow. 

Cone  of  courses  high  speed  leg  [bearing?]  {tack}  [speed]. 

Low  speed  leg  [bearing]  {tack}  [speed]. 

Sector  assignment  [bearing]  to  [bearing]  desig  [call  sign], 

ETA/echo_tango_alpha  TDA/tango_deltav_alpha  [time]. 

Execute  to  follow  turn  [alpha/bravo]  desig  [alpha/bravo] 
[course]. 

I  am  brother/sister. 
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TABLE  u-10.  PHRASES  USED  BY  THE  HELICOPTER  PILOT. 


Circuit  Phrase 


ASW  Helo  Control  On  your  command. 

Net 

Hike  speed  [speed]  cherubs  [digit]. 

King  pin. 

Spit  one  maypole. 

Road  complete. 

Conducting  M2  [compass  point]  present  position. 
MADman  MADman  MADman. 

No  joy  this  pass. 

[speed]  buster. 

I  am  buster, 

w - 


EVALUATION  OF  AUTOMATED  SPEECH  HARDWARE 

Although  it  is  not  within  the  scope  of  this  project  to  perform  trade¬ 
off  studies  to  determine  the  best  automated  speech  hardware  for  the  training 
system,  the  questions  given  in  Tables  G-11  and  G-12  provide  a  starting  point 
for  performing  such  an  evaluation. 
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TABLE  G— 11-  SPEECH  RECOGNITION  DEVICE  CHECKLIST. 


Does  the  device  recognize  isolated  words  or  phrases,  or  continuous  speech? 

What  stylization  constraints  are  imposed? 

How  long  an  utterance  can  be  accepted? 

How  is  recognition  accuracy  affected  by  having  both  relatively  3hort  and 
relatively  long  phrases  in  the  vocabulary? 

What  is  the  maximum  vocabulary  size  the  device  can  recognize? 

Does  recognition  accuracy  vary  between  male  and  female  talkers? 

What  is  the  recognition  accuracy  for  the  vocabulary  needed  for  *y  application? 
How  is  recognition  accuracy  related  to  vocabulary  size? 

How  fast  is  speech  recognition? 

How  is  speech  recognition  rate  related  to  vocabulary  size? 

Can  the  recognition  unit  be  directed  to  restrict  its  search  to  a  few  items 
in  the  vocabulary?  (Partitioning  the  vocabulary  can  improve  recognition.) 
Does  the  unit  supply  a  measure  of  confidence  in  the  recognition  result? 

Does  it  provide  a  second  and  even  third  choice  when  it  is  not  sure  what  was 
spoken? 

Can  the  threshold  for  supplying  a  second  choice  be  adjusted? 

Can  the  threshold  for  whether  or  not  recognition  has  occurred  be  adjusted? 

(In  some  applications,  any  clue  to  what  was  spoken  is  helpful,  thus 
one  threshold  is  needed.  In  other  applications,  it  is  necessary  to 
minimize  the  possibility  of  a  recognition  error,  hence  a  different  threshold 
is  appropriate.) 

What  is  the  effect  of  background  noise  on  the  system? 

Can  it  recognize  speech  over  the  telephone? 

Does  the  unit  supply  an  indication  of  voice  level  with  the  recognition? 

(Poor  microphone  placement,  common  with  the  naive  user,  can  lead  to 
poor  recognition.  With  an  indication  of  voice  level,  the  system  has 
a  clue  to  the  reason  for  poor  recognition  and  can  provide  helpful  feedback. 
At  present,  this  feature  is  not  known  to  be  commercially  available, 
and  has  had  to  be  implemented  as  a  special  purpose  device  for  our  own 
applications. ) 

Is  the  device  speaker  dependent? 

If  so,  how  many  times  does  each  word  have  to  be  repeated  in  order  to  configure 
the  nystem  to  recognize  an  individual  talker? 

How  is  recognition  accuracy  related  to  number  of  repetitions? 

Can  the  data  for  an  individual  talker  be  saved  on  an  easily  accessible  external 
medium,  or  does  the  system  have  to  be  reconfigured  each  time  it  is  used? 
Can  the  configuration  process  be  tailored  to  the  needs  of  the  individual 
applies. Lon? 

Can  a  few  items  be  trained  at  a  time? 

Can  the  order  of  presentation  be  varied  or  does  the  talker  have  to  repeat 
each  item  a  number  of  times  in  succession? 

Can  a  single  speech  pattern  be  updated  easily? 
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TABLE  G— 11 .  SPEECH  RECOGNITION  DEVICE  CHECKLIST,  CONT. 


How  much  host  computer  memory  is  needed  for: 

Speech  system  configuration  and  update  procedures? 

Speech  recognition  procedures? 

Speech  data  storage  (raw  input  data  and  voice  patterns)? 

How  does  the  host  computer  communicate  with  the  device? 

What  is  the  I/O  processing  burden? 

What  type  of  interface  is  provided? 

Is  the  data  transfer  rate  sufficient? 

How  long  does  it  take  to  transfer  speech  patterns? 

How  long  does  it  take  to  transfer  recognition  information? 

How  sensitive  to  damages  are  the  components  which  the  user  carries  or  wears? 

(Some  noi3e  canceling  microphones  can  be  irreparably  damaged  by  dropping 
them  from  even  a  short  distance.  Users  will  drop  them  and  also  accidentally 
tug  on  the  cords,  etc.) 


TABLE  G-12.  SPEECH  GENERATION  SYSTEM  CHECKLIST. 


How  much  time  is  required  to  create  the  required  vocabulary? 

(Even  "text  to  speech"  devices  require  some  hand  smoothing  of  the  phrases 
to  achieve  good  inflection,  etc.) 

How  easy  is  it  to  change  the  vocabulary? 

(Some  devices  require  encoding  by  the  manufacturer. ) 

To  what  extent  can  individually  stored  words  or  phrases  be  put  together  dynam¬ 
ically  to  produce  real-time  output  in  response  to  a  changing  situation? 
(The  resulting  speech  is  usually  very  choppy.) 

How  many  different  voices  can  be  simulated? 

Is  speech  rate  under  computer  control? 

How  fast  can  the  device  speak? 

Is  pitch  under  computer  control? 

Does  the  device  notify  the  computer  that  speech  output  is  complete? 

Does  the  device  require  a  data  storage  srea  to  be  provided  by  the  host  computer? 
If  so,  approximately  how  many  words  of  data  storage  are  required  per  word 
or  per  second  of  speech? 

What  I/O  burden  does  the  system  place  on  the  computer? 
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APPENDIX  H 

SOFTWARE  DESIGN  CONSIDERATIONS 

This  section  addresses  itself  to  the  software  design  issues  which  will 
significantly  influence  the  development  and  evolution  of  the  team  training 
demonstration  system.  The  principal  design  issues  which  will  be  discussed 
below  are  simulation,  speech,  and  software  architecture. 

SIMULATION  METHODS 

According  to  the  American  Heritage  Dictionary,  the  word  simulate  means 
"to  have  or  take  on  the  appearance,  form,  or  sound  of;  to  imitate. n  This 
definition  is  just  possibly  broad  enough  to  cover  the  realm  of  things  which 
simulation  software  attempts.  Historically,  simulation  software  was  first 
created  to  model  deterministic  physical  processes  and  engineering  structures; 
later  probabilistic  methods  (e.g.,  Monte  Carlo  schemes)  were  created  to  model 
situations  too  complex  for  the  earlier  deterministic  approaches.  In  the 
last  decade  or  so,  simulation  has  developed  into  a  much  broader  v.jtion,  wherein 
the  something  which  is  imitated  may  include  complex  human  behavior  and  sophis¬ 
ticated  decision  making. 

It  is  pretty  obvious  that  simulation  methods  which  are  appropriate  to 
model  a  physical  process  may  well  be  inadequate  to  represent  complex  human 
behavior.  Simulations  of  the  consistency  of  mud  for  oil  drilling,  of  high 
performance  aircraft  for  durability  testing,  of  wheat  price  fluctuations 
for  economic  forecasting,  and  of  human  performance  in  high  pressure  decision 
making  environments  may  be  qualitatively  quite  different.  In  the  matter 
of  simulation  languages,  for  example:  FORTRAN  is  a  very  convenient  vehicle 
for  the  simulation  of  physical  processes.  It  has  some  undesirable  features 
for  the  representation  of  complex  mathematics,  but  from  a  practical  standpoint 
it  is  really  quite  functional  (much  more  so  than  its  many  vocal  oritics  would 
admit).  On  the  other  hand,  representations  of  human  behavior  -  or  most  anything 
else  that  is  not  number  (or  data)  intensive  tends  to  be  rather  awkward  in 
FORTRAN.  Other  languages  and  systems  have  stepped  in  to  fill  the  void. 
Foremost  among  them  has  been  the  LISP  family.  And  LISP  in  turn  has  given 
impetus  to  a  whole  variety  of  the  so  called  list-oriented  or  logic  programming 
languages. 

"Thirty  days  hath  September,  April,  June,  and  November."  How  might 
two  different  simulations  languages  represent  this  set  of  data? 

A  FORTRAN  programmer  might  well  use  DATA  statements: 

DATA  MONTH  (4)/30/ 

DATA  MONTH  (6)/30/ 

DATA  MONTH  (9)/30/ 

DATA  MONTH  (11)/30/. 
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A  programmer  using  the  MACLISP-base d  production  rule  language  called 
RITA  might  code  this  information  as  follows: 

If  the  month  is  April,  June,  September,  or  November, 
then  the  number  of  days  is  30. 

Cf  course,  there  is  a  large  expressive  range  in  both  languages,  but 
these  are  representative  means  of  encoding  our  information.  The  RITA  version 
is  verbose  in  some  sense,  but  the  FORTRAN  version  -  to  the  uninitiated,  at 
least  -  is  stark  and  forbidding.  A  non-programmer  would  probably  find  the 
RITA  statement  perfectly  understandable  and  virtually  indistinguishable  from 
English.  However,  this  non- programmer  may  not  be  able  to  make  any  sense 
at  all  of  the  FORTRAN. 

This  example  seems  to  stack  the  cards  against  FORTRAN.  But  there  are 
many  instances  wherein  FORTRAN  presents  affective  and  readable  commands. 
For  instance,  the  oommand  to  compute  a  new  sonar  signal  excess  in  FORTRAN 
is  expressed  easily  in  one  line  which  parallels  nicely  the  more  conventional 
mathematical  expression. 

It  must  be  emphasized,  though,  that  the  matter  of  programming  languages 
is  entirely  secondary  and  that  only  symptoms  of  deeper  issues  are  revealed 
in  something  like  the  example  we  discussed  above.  What  is  the  oentral  issue? 
What  really  distinguishes  one  kind  of  simulation  method  from  another? 

Terry  Winograd,  in  an  article  entitled  "Toward  Convivial  Computing, "2 
argues  that  the  whole  of  computer  technology  has  developed  in  three  distinct 
stages.  The  first  and  earliest  stage  concentrated  on  the  number  crunching 
functions  of  the  computer  and  computer  resolution  of  immense  numerical  problems. 
The  second  stage  was  the  development  of  the  idea  of  data  processing  and  of 
the  capability  to  establish  and  manipulate  data  bases  whose  elements  include 
much  that  is  not  strictly  numerical.  The  third  stage,  one  which  computer 
technology  is  just  now  entering,  is  the  era  of  knowledge  engineering.  With 
this  technology  it  is  the  chunk  of  knowledge,  as  opposed  to  the  individual 
number  or  piece  of  data,  which  is  manipulated  to  generate  new  information 
and  new  knowledge. 

Characteristic  of  knowledge  engineering  is  a  greater  concern  for  the 
manner  in  which  knowledge  gets  into  a  system,  the  internal  form  it  assumes 
within  the  computer,  and  the  shape  in  which  it  is  presented  to  a  computer 
system  user.  Ideally,  the  principles  of  knowledge  engineering  promise  to 
promote  a  much  smoother  and  more  graceful  interaction  between  computer  system 
and  the  unsophisticated  user. 


2.  The  Computer  Age:  A  Twenty.  Year  View.  M.  L.  Dertouzous  and  J.  Moses 
(editors)  MIT  Pre3»,  1979,  p.  56-72 
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And  what  of  simulation  methods?  To  a  great  extent  the  development  of 
simulation  methods  has  followed  the  course  described  by  Winograd.  The  different 
simulation  methods  are  different  by  virtue  of  the  raw  material  with  which 
they  work.  Physical  process  simulations  naturally  operate  on  numbers;  complex 
simulations  of  social  and  economic  interactions  use  items  of  information 
from  complex  data  bases;  simulations  of  more  complex  individual  human  behavior 
are  naturally  knowledge-based.  The  knowledge  required  of  a  human  behavior 
simulation  can  indeed  be  quite  complicated:  it  may  include  both  the  explicitly 
known  and  the  implicitly  held,  the  experimental,  the  intuitive,  and  the  de¬ 
ductive. 

But  knowledge  engineering  techniques  have  the  potential  to  get  access 
to  and  use  profitably  many  of  the  different  varieties  of  human  knowledge 
which  more  traditional  methods  of  ctmulntion  might  not  be  able  to  use,  or 
might  be  able  to  use  but  only  in  painfully  ad  hoc  ways.  The  simulation  method 
should  be  appropriate  to  the  atoms  with  which  the  simulation  works:  if  knowledge 
is  the  raw  material,  why  not  use  knowledge  engineering  techniques?  If  numbers 
are  the  raw  stuff,  then  use  existing  mathematical  modeling  techniques. 

TRADITIONAL  SIMULATIONS  FOR  TEAM  TRAINING 

For  the  team  training  demonstration  system,  the  functional  requirements 
of  several  of  the  simulation  subsystems  clearly  call  for  a  classical  mathematical 
modeling  approach.  For  sonar,  radar,  and  magnetic  anomaly  detection  the 
safest,  most  efficient,  and  most  economical  means  of  handling  the  simulation 
is  to  code  the  set  of  equations  which  model  the  phenomena  in  some  language 
like  FORTRAN.  For  ASW  support  personnel,  the  oase  for  classical  simulation 
Is  not  quite  so  strong,  but  since  basically  data  and  message-handling  activities 
are  irvolved,  and  since  the  support  models  will  all  be  rather  small,  it  will 
be  most  convenient  to  use  a  simple  simulation  of  a  kind  of  message  passing 
and  message  handling  network.  For  the  team  members,  something  much  different 
is  oalled  for.  Just  what  this  might  be  is  disoussed  in  the  next  section. 

KNOWLEDGE-BASED  SIMULATION  FOR  TEAM  TRAINING 

Confronted  with  a  strong  sonar  contact  with  an  apparently  hostile  submarine, 
the  ASWOC  must  make  several  decisions  in  short  order.  He  must,  for  instance, 
issue  a  command  for  his  ship  to  begin  to  execute  evasive  maneuvers  before 
it  is  in  danger  from  the  submarine.  In  the  team  training  demonstration  system, 
this  decision  point  will  surely  come  up  again  and  again  and,  if  the  ASWOC 
is  missing,  the  ASWOC  simulation  subsystem  itself  must  recognize  the  situation, 
make  a  decision,  and  issue  the  command.  How  might  this  be  done? 

One  very  appealing  approach  to  this  problem  uses  a  production  system 
based  on  production  rules.  For  this  example,  the  relevant  production  rule 
would  be  something  like: 

If  sonar  contact  with  a  hostile  is  made,  and 

if  the  oontact  is  X  miles  from  ownship, 

then  ASWOC  should  recommend  evasive  activity. 
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(The  precise  form  of  this  rule,  because  it  represents  a  taotic,  is  classified, 
so  this  rule  is  more  vague  than  it  would  appear  in  the  real  system.)  One 
advantage  of  the  production  rule  representation  is  immediately  apparent; 
production  rules  make  sense  to  people  other  than  programmers.  A  subject 
matter  expert,  with  minimal  effort,  should  be  able  to  understand  all  the 
production  rules  in  the  form  they  are  used  by  the  program. 

Another  advantage  of  the  knowledge-based  simulation,  as  exemplified 
by  the  production  system,  is  that  individual  bits  of  information  need  only 
be  coded  once.  Our  rule  for  the  ASWOC  should  serve  to  aid  the  ASWOC  model, 
should  provide  information  for  the  performance  measurement  system,  and  should 
serve  to  predict  for  the  speech  understanding  system  just  what  the  ASWOC 
might  say  next.  All  this  from  one  rule  in  one  place. 

Furthermore,  the  rule-based  simulation  should  aid  in  annotating  and 
explaining  team  member  errors.  For  instance,  had  the  ASWOC  failed  to  recommend 
commencing  evasive  maneuvers,  the  PM  system  would  recognize  this,  and  note 
that  the  ASWOC  failed  to  respond  according  to  rule  #  N. 

Of  course,  there  are  potential  disadvantages  to  a  knowledge-based  approach. 
These  are  described  in  Table  H-1 .  There  is  definitely  an  element  of  technical 
risk  involved  in  knowledge-based  simulation  at  the  present  time,  Hut  what 
of  the  alternative?  Traditional  simulation  methods  are  better  known  and 
better  supported  and  are  thus,  in  some  sense,  less  risky,  but  there  is  a 
serious  question  whether  these  older  simulation  methods  are  adequate  to  the 
task  of  modeling  team  member  performance. 

SOME  PRECEDENTS  FOR  KNOWLEDGE-BASED  SIMULATION 

In  the  last  several  years,  several  knowledge-based  systems  have  been 
designed  and  built,  and  many  are  now  functioning.  Early  breakthroughs  in 
this  area  were  made  at  Stanford  University  where  a  series  of  expert  knowledge- 
based  programs  developed  from  an  initial  seed. 

The  MYCIN  system  was  developed  by  Randall  Davis,  Bruoe  Buchanan,  and 
Edward  Shortliffe  to  provide  consultative  advice  on  the  diagnosis  of  and 
therapy  for  infectious  diseases  -  in  particular,  bacterial  diseases  in  the 
blood.  MYCIN  is  a  high  performance  program  which  has  succeeded  in  attracting 
the  interest  and  assistance  of  medical  experts  all  over  the  world. 

DENDRAL,  developed  by  Bruce  Buchanan  and  Joshua  Lederberg,  is  an  expert 
program  designed  to  aid  in  the  structural  analysis  of  oomplex  molecules  by 
creating  and  testing  hypotheses  based  on  raw  mass  speotro3oopy  data.  This 
program  is  good  enough  so  that  it  has  done  "original  research"  in  the  sense 
that  it  has  produced  publishable  results. 
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Table  H-1.  ADVANTAGES  AND  DISADVANTAGES  OF  KNOWLEDGE- BASED  SIMULATION. 


_ A&an&aggfi _ 

Permits  orderly  and  unified 
development  of  large  amounts  of  a 
task  specific  information. 

Facilitates  extraction  of  information 
from  subject  matter  experts,  as  well 
as  examination  and  correction  of 
previously  encoded  knowledge. 

Allows  relatively  rapid  modification 
to  the  knowledge  base  by  changing 
rules,  adding  rules,  or  deleting 
rules. 

Operates  well  in  an  interactive  mode 
In  explaining  decisions,  calling  for 
new  data,  modifying  existing  rules: 
capable  of  being  very  supportive  of 
PM. 

Can  explain  a  ohain  of  reasoning  and 
conclusions  reached.. 

Permits  "chunking”  of  knowledge  so 
that  each  rule  is  a  comprehensible 
statement  of  some  piece  of  system 
knowledge. 

Provides  capability  for  dealing  with 
Judgmental  knowledge  and  probabilis¬ 
tic  reasoning. 

Should  (after  initial  learning 
experience)  aid  coding  and  debugging 
considerably. 

Provides  capability  to  include  meta¬ 
rules  in  a  natural  way  to  influence 
strategy  and  directions  of  reasoning. 

Promotes  smooth  development  from 
a  working  skeleton  to  full  version. 


_ Bjjga&antflgsfl _ 

Requires  time  consuming  work  with 
subject  matter  experts  to  extract 
relevant  knowledge  and  to  formalize 
it.  Difficulty  encountered  by  SME's 
in  ooping  with  this  formalization, 
as  well  as  difficulty  in  dealing 
with  the  necessary  introspection. 

Potentially  large  memory  resources 
are  called  for. 

Real  time  operation  is  ohanoey. 

Production  rule  format  is  not  natural 
in  some  oases;  knowledge  of  some 
kinds  is  not  naturally  expressed 
this  way. 

Large  number  of  potential  production 
rules  are  involved  -  especially 
if  three  team  members  are  to  be 
modeled.  Overlap  of  knowledge 
ameliorates  this  somewhat,  but  how 
muoh? 

Inference  procedures,  especially 
ones  like  backward  chaining,  occasionally 
involve  long  chains  of  complicated 
reasoning.  Perhaps  difficult  to 
track  down  cause  of  errors. 
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TEIRESIAS,  developed  by  Randall  Davis,  was  designed  to  offer  assistance 
in  the  interactive  transfer  of  knowledge  from  a  human  expert  to  the  knowledge 
base  of  a  high  performance  program.  Originally  developed  for  use  in  conjunction 
with  MYCIN,  TEIRESIAS  has  proven  to  be  significantly  more  versatile.  For 
example,  TEIRESIAS  also  has  acted  as  the  knowledge  acquisition  front  end 
for  an  investment  advisory  program. 

GUIDON,  developed  by  W.  J.  Clanoey,  is  a  case  method  tutoring  program 
designed  to  improve  a  student's  ability  to  diagnose  complex  problems  in  a 
scientific  or  medical  domain.  GUIDON  differs  from  other  computer-aided  instruc¬ 
tional  programs  in  that  it  has  domain  -  independent  rules  which  guide  the 
presentation  of  materials.  Furthermore,  GUIDON  teaches  knowledge  in  terms 
of  rules  of  good  judgement,  as  well  as  factual  knowledge. 

Of  course,  a  lot  of  good  work  has  been  done  in  places  other  than  Stanford. 
The  RAND  Corporation,  for  example,  has  developed  two  more-or-less  general 
purpose  production  system  languages  called  ROSIE  and  RITA.  Philip  Klahr 
and  William  S.  Faught  at  RAND  have  developed  ROSS  (Rule-Oriented  Simulation 
System)  which  simulates  the  tactical  environment  of  air  warfare  in  a  rather 
sophisticated  way. 

At  the  Naval  Ocean  Sea  Center  (NOSC),  Dennis  C.  McCall  and  Robert  J.  Bechtel 
developed  STAMMER,  a  production  rule-based  system  designed  to  simulate  the 
identification  of  unknown  ships  at  sea. 

The  examples  cited  here  are  just  a  small  sample  indicating  the  level 
of  activity  in  the  area  of  expert  systems  and  knowledge-based  simulation. 

A  SAMPLE  SOFTWARE  ARCHITECTURE  FOR  A  KNOWLEDGE- BAS ED  SYSTEM 

One  kind  of  knowledge-based  simulation  system  which  might  be  used  to 
model  the  ASW  team  members  is  described  below  in  a  sketchy  way.  The  approach 
taken  here  is  to  use  a  production  rule-based  system,  but  other  reasonable 
choices  are  possible. 

What  kinds  of  knowledge  must  the  system  incorporate?  Basically,  there 
are  three  varieties:  declarative,  procedural,  and  control  knowledge.  These 
three  flavors  of  knowledge  combine  to  form  a  generalized  production  system 
in  the  following  way: 

1.  The  repository  of  declarative  knowledge  is  a  global  data  base.  This 
data  base  holds  statements  which  have  the  basic  form:  An  k  is  a  y  with 
properties  p  and  q.  For  example,  "the  ocean  conditions  are  characterized 
by  Beaufort  wind  scale  value  v  and  sea  state  s." 

2.  -The  procedural  knowledge  is  represented  by  a  set  of  production  rules 

consisting  of  preconditions  required  to  satisfy  the  hypotheses  of  rules 

followed  by  the  consequenoes  of  the  hypotheses.  When  a  rule's  hypotheses 

are  satisfied,  the  rule  is  activated.  For  example,  a  typical  ASW  production 
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rule  might  have  the  form: 

If  the  weapon's  danger  area  is  not  clear  of  friendly  foroes, 
then  the  weapon  should  not  be  fired. 

When  the  hypothesis  is  met  (i.e.,  there  is  a  friendly  craft  in  the  way), 
the  rule  is  activated  and,  in  the  absenoe  of  other  information,  the  system 
will  deoide  that  no  weapon  should  be  fired. 

3.  Control  knowledge  is  embodied  in  a  control  system  whioh  selects  activated 
rules,  applies  them,  and  modifies  the  data  base  accordingly.  The  select/ 
apply  oycle  is  repeated  until  some  termination  condition  on  the  data 
base  is  met. 

Perhaps  the  most  subtle  of  these  three  areas  is  control.  There  are 
a  variety  of  control  strategies  whioh  might  be  employed  to  create  the  so-called 
inference  engine.  Control  strategy  sohemes  can  be  tentative  or  irrevocable 
(can  or  oannot  back  up  from  and  reoonsider  previous  decisions),  forward  or 
backward  (proceeding  to  goal  state,  or  backward  from  goal  to  subgoals),  decom¬ 
posable  or  lnde 000 posable  (data  base  does  or  does  not  have  separate  components 
which  can  be  handled  individually) ,  and  order  dependent  or  independent  (order 
in  which  produots  or  rules  are  applied  is/is  not  relevant). 

The  design  of  an  adequate  knowledge-based  modeling  framework  for  team 
training  requires  that  the  ohoices  of  control  strategies  be  made  as  best 
possible  to  support  ASW  training. 
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APPENDIX  I 

A  KNOWLEDGE-BASED  MODEL  OF  THE  ASWOC 
1..  PRODUCTION  AND  BACKGROUND 

The  purpose  of  this  appendix  is  to  sketch  a  design  for  a  knowledge-based 
simulation  of  the  ASWOC' s  activity  as  part  of  an  ASW  team  in  aotive  pursuit 
of  a  submarine.  While  the  focus  of  most  work  done  under  the  Team  Training 
Through  Communications  Control  contract  has  been  on  three  member  teams,  our 
emphasis  in  this  appendix  will  be  to  look  at  an  approach  for  modeling  just 
one  member  of  that  team.  We  restrict  our  attention  in  this  way  because  it 
is  important,  first  of  all,  to  see  if  a  knowledge-based  simulation  is  feasible 
for  one  t  am  member  before  committing  the  resources  to  design  models  for 
all  the  team  members.  Secondly,  available  resources  at  this  time  were  sufficient 
only  for  examining  one  model  in  detail.  Thirdly,  the  approach  we  have  chosen 
uses  a  structure  which  appears  to  be  applicable  for  the  other  team  members 
as  well.  Thus,  the  basic  design  we  present  would  almost  certainly  provide 
the  fundamental  structures  to  support  a  complete  design  for  each  team  member 
model. 


We  focused  on  the  ASWOC  because  the  modeling  cf  the  ASWOC's  activities 
seems  to  require  more  complexity  and  more  variety  than  either  the  ASAC  or 
the  ASWFCO  models.  In  part,  this  is  because  the  ASWOC's  job  calls  for  a 
greater  degree  of  high-level  cognitive  thinking  and  decision  making,  and 
requires  as  well  knowledge  of  a  significant  collection  of  procedural  and 
manipulative  skills.  We  chose  the  most  challenging  model  to  start  with  because, 
at  this  stage  of  development,  it  is  important  to  te3t  a  knowled^'  jased  design 
against  a  complex  facet  of  the  modeling  problem  to  see  if  the  approach  is 
workable,  and  to  get  an  idea  of  its  potential  complexity. 

To  a  certain  extent,  we  have  limited  the  modeling  problem  by  restricting 
ourselves  to  that  aspect  of  the  ASWOC's  behavior  involved  with  choosing  and 
firing  a  weapon  at  a  known  submarine  in  a  known  position,  and  then  retiring. 
This  limitation  does  not  apply  tc  the  structural  design  of  the  ASWOC  model, 
which  should  be  capable  of  supporting  any  aspect  of  the  ASWOC's  ASW  activity. 
The  restriction  does  apply  to  the  specific  enumeration  of  individual  bits 
of  knowledge  which  the  ASWOC  will  need;  to  get  an  idea  of  the  magnitude  of 
this  detailed  knowledge,  we  assembled  knowledge  "particles''  for  this  limited 
part  of  the  ASW  problem  to  present  a  sample  of  the  basic  knowledge  structure. 

The  design  which  is  presented  in  the  following  pages  has  been  developed 
in  the  spirit  of  knowledge-based  systems  engineering.  The  basic  structure 
of  the  design  is  a  modified  production  rule  system  which  is  capable  of  operating 
and  communicating  with  an  external  system  which  regards  the  ASWOC  model  as 
just  one  of  its  own  subsystems.  For  our  purposes,  we  shall  concentrate  on 
the  ASWOC  model  per  se,  and  assume  that  the  rest  of  the  system  is  defined. 
Where  it  is  necessary  or  illuminating,  we  will  include  some  details  of  the 
interface  between  the  ASWOC  model  and  the  external  system. 
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CHOICE  OF  A  KNOWLEDGE- BASED  STRUCTURE 

WHY  A  KNOWLEDGE-BASED  STRUCTURE?  A  number  of  factors  were  involved  in  our 
deoision  to  model  the  ASWOC's  behavior  with  knowledge  engineering  techniques 
as  opposed  to  more  traditional  procedural  simulation  techniques.  (A  rprooadural 
simulation  technique"  is  one  which  represents  each  activity  or  process  to 
be  simulated  as  a  strict  sequence  of  actions  to  be  followed,  one  after  another, 
possibly  contingent  on  intervening  decision  logic  to  determine  which  of  several 
sequences  is  to  be  followed.  A  knowledge-based  simulation  technique,  in 
contrast,  does  not  pre-define  a  sequenoe  of  actions  to  be  followed.)  Our 
reasons  for  following  the  knowledge  engineering  approach  are  described  in 
detail  in  other  sections  of  this  report,  but  we  will  summarize  the  main  points. 

Knowledge-based  systems: 

a.  Collect,  retain,  and  use  knowledge  relevant  to  system  functioos 
in  a  manner  which  is  much  closer  to  human  functioning  than  traditional 
methods.  Thus  the  basic  structure  is  much  more  comprehensible 
to  human  non-specialists. 

b.  Make  it  possible  to  code  significant  knowledge  Just  once  for 
use  in  several  contexts.  This  contributes  to  interned  consistency, 
ease  of  design  and  maintenance,  greater  comprehensibility,  smoother 
development,  and  facilitated  correction  of  errors. 

c.  Are  capable  of  re-tracing  their  operations  and  explaining  how 
their  decisions  are  arrived  at.  This  makes  debugging  of  the  systems 
significantly  easier  at  all  levels. 

d.  Greatly  facilitate  an  incremental  developmental  process  whereby 
an  initial  functioning  core  can  be  fleshed  out  gradually,  with 
new  capabilities  and  new  knowledge  added  piece  by  pleoe. 

The  advantages  of  using  a  knowledge-based  approach  to  model  the  ASWOC 
are  considerable.  The  ASWOC's  behavior  is  complex,  and  traditional  means 
of  simulating  his  activities  using  a  basically  procedural  approach  would 
probably  produce  a  very  large  body  of  unwieldy  code  which  would  be  hard  to 
debug  and  harder  still  to  maintain.  A  knowledge-based  approaoh  offers  the 
capability  to  cut  through  the  complicated  interactions  inherent  in  the  procedural 
approach  by  lotting  the  system  respond  dynamically  to  the  environment  as 
it  changes. 

WHAT  KIND  OF  A  KNOWLEDGE- BASED  SYSTEM?  Once  we  had  made  the  decision  to 
take  a  knowledge-engineering  approach  to  the  problem  of  modeling  the  ASWOC, 
we  had  to  decide  what  kind  of  a  knouledge-based  system  paradigm  was  best 
for  the  ASW  team  training  situation.  Over  the  past  fifteen  years,  a  number, 
of  generically  different  approaches  to  knowledge-based  programming  have  devel* 
oped.  Generally,  these  approaches  have  evolved  around  certain  kinds  of  problems 
(e.g. ,  cognitive  modeling,  understanding  stories,  simulating  short  term  memory, 
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modeling  the  know-how  of  an  expert,  etc.).  It  was  apparent  from  the  beginning 
that  one  kind  of  knowledge-based  formalism  was  an  especially  good  candidate 
for  ASW  team  member  modeling,  but  we  considered  other  possibilities  carefully 
to  assure  ourselves  that  our  initial  predilection  had  an  adequate  foundation 
in  theoretical  foundation.  We  chose  then  to  proceed  with  our  original  favorite 
and  use  a  production  rule  based  system  (production  system  for  short)  to  model 
the  ASWOC.  Our  justification  for  this  choice  will  be  explained  later,  after 
we  have  sketched  a  description  of  a  generic  production  system. 

Production  systems  were  introduced  by  the  mathematician  Emil  Post  in 
1943  (see  reference  [4]).  For  Post,  a  production  system  was  a  general  compu¬ 
tational  mechanism  which  had  strong  potential  for  the  study  of  formal  systems. 
Since  their  introduction,  production  systems  have  undergone  a  great  deal 
of  development,  their  methodology  has  expanded  considerably,  and  they  have 
been  applied  to  an  extremely  diverse  collection  of  problems. 

A  pure  production  system  may  be  looked  at  as  a  combination  of  three 
basic  components:  a  set  of  rules,  a  data  base,  and  an  interpreter  for  the 
rules.  In  the  purest  and  simplest  design,  a  rule  is  an  ordered  pair  of  symbol 
strings  with  a  left-hand-side  and  a  right-hand-side;  the  rule  set  has  a  pre¬ 
determined  total  ordering;  and  the  data  base  is  just  a  collection  of  symbols. 
In  this  simple  design,  the  interpreter  works  by  scanning  the  left-hand-side 
of  each  rule  in  turn  until  one  is  found  which  matches  some  element  of  the 
data  base.  At  that  point,  the  symbols  matched  in  the  data  base  are  replaced 
with  those  found  in  the  right-hand-side  of  the  applicable  rule,  and  scanning 
either  continue:  with  the  next  rule  or  begins  again  from  the  first.  The 
rule  selection  function  can  also  be  viewed  as  a  chained  sequence  of  modus 
oonens  operations. 

To  put  some  flesh  on  these  bare  bones,  consider  the  following  example. 
Let  there  be  three  production  rules  in  our  system: 

PI.  If  there  are  ducks  on  the  pond,  then  the  fish  are  uneasy. 

P2.  If  the  fish  are  uneasy,  then  it  is  raining. 

P3.  If  it  is  raining,  then  there  are  ducks  on  the  pond. 

Suppose  our  data  base  consists  initially  of  the  string  "it  is  raining". 
Further,  assume  that  the  rule  interpreter  is  set  up  to  apply  whatever  rules 
are  applicable  on  each  cycle,  and  to  replace  the  current  data  base  entry 
with  the  right-hand-side  of  the  applicable  rule. 

After  the  first  rule  interpretation  cycle,  rule  P3  has  been  applied, 
and  the  data  base  holds  "there  are  ducks  on  the  pond."  After  the  second 
cycle,  rule  Pi  has  been  applied  and  the  data  base  holds  "the  fish  are  uneasy." 
At  the  end  of  the  third  cycle,  rule  P2  has  been  applied,  and  the  data  base 
once  again  holds  "it  is  raining."  Clearly,  this  pattern  is  repeated  indefinitely 
in  this  example.  Note  that  the  production  rules  can  be  considered  formally 
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as  left-hand-side,  right-hand-side  ordered  string  pairs— e.g„,  it- is- raining, 
there-are-ducks-on-the-pond — or  as  ordinary  conditional  statements  of  the 
form  condition-action  or  hypothesis-conclusion.  Also,  notice  that  the  data 
base  can  be  thought  of  as  the  state  of  the  world  at  any  given  instant. 

We  will  describe  the  production  system  architecture  we  have  chosen  for 
the  ASWOC  model  in  detail  in  the  next  section  of  this  appendix.  Since  it 
is  not  a  pure  production  system,  we  will  discuss  at  the  same  time  those  few 
additional  structures  we  need  to  make  the  model  complete.  In  the  remainder 
of  this  section,  we  will  address  the  issue  of  the  appropriateness  of  a  production 
system  for  modeling  the  ASWOC. 

WHY  PRODUCTION  SYSTEMS  PROVIDE  A  GOOD  PROTOTYPE.  Program  designers  have 
found  that  production  systems  easily  model  problems  in  some  domains,  uu! 
are  awkward  in  others.  From  a  purely  theoretical  point  of  view,  production 
systems  and  traditional  procedural  systems  are  formally  equivalent,  because 
booh  in  turn  are  formally  equivalent  to  a  simple  Turing  machine.  So  the 
question  is  not  which  approach  can  do  the  job  (since  if  one  can,  they  both 
can) ,  but  instead  which  approach  offers  the  greater  economy  of  design,  ease 
of  implementation,  and  gracefulness  of  operation.  Randall  Davis  and  Jonathon 
King  surveyed  the  breadth  of  applications  of  production  systems  (1977)  »  and 
they  offered  some  observations  about  problem  domains  in  vbicn  production 
systems  could  be  most  effectively  used.  In  their  view,  prod.vtion  systems 
appear  to  be  most  useful  where  it  is  important  to  detect  and  deal  with  a 
potentially  large  number  of  independent  states,  in  a  system  which  requires 
a  broad  scope  of  attention,  and  the  capability  of  reacting  swiftly  to  small 
changes.  In  addition,  they  note  that  where  knowledge  of  the  problem  domain 
falls  naturally  into  a  sequence  of  relatively  independent  "recognize-act" 
pairs,  production  systems  offer  a  convenient  formalism  for  structuring  and 
expressing  that  knowledge. 

The  modeling  of  the  ASWOC,  ASAC,  and  ASWFCO  seems  to  fit  very  naturally 
into  a  production  rule  system  format.  To  be  sure,  a  number  of  additions 
and  modifications  to  the  pure  production  system  structure  appear  to  be  necessary, 
but  the  basic  structure  is  more  than  compatible  with  the  requirements  and 
the  demands  which  these  models  make.  If  we  focus  once  again  on  the  ASWOC 
model,  we  can  point  to  some  specific  reasons  why  the  production  system  idea 
is  sc  appropriate. 

The  two  features  of  a  production  system  which  make  it  suitable  for  modeling 
the  ASWOC  are  the  simple  and  practical  form  of  the  rule  structure  and  the 
openness  of  the  control  structure.  The  rule  structure  seems  very  nearly 
ideal  for  representing  the  ASWOC' s  knowledge.  A  great  deal  of  the  ASWOC’ s 
know-how  is  most  naturally  expressed  in  condition-action  form.  Furthermore, 
the  dynamic  nature  of  the  ASWOC' s  environment  dictates  an  approach  which 
is  responsive  and  reactive:  there  are  a  very  largo  number  of  possible  changes 
in  the  environment,  and  they  must  be  responded  to  actively  in  a  short  period 
of  time.  In  addition,  instruction  manuals  typically  convey  informatiosi  in 
a  form  which  is  akin  to  that  of  production  rules,  and  a  functioning  ASWOC 


NAVTRAEQUIPCEN  80-C-0095-1 


I 

""  also  tends  to  describe  his  functions  in  those  terms.  The  real  test,  however, 

comes  when  a  subject  matter  expert  tries  to  reduce  his  complex  knowledge 
of  the  ASWOCs  job  into  production  rules. 


Via  performed  this  test  for  the  small  segment  of  the  ASW  problem  which 
we  have  already  described,  and  the  production  rules  which  evolved  are  shown 
In  Appendix  D.  The  subject  matter  expert  had  some  difficulty  in  describing 
the  ASWOC’s  functions  at  such  a  basic  level,  but  several  sessions  with  a 
knowledge-expert  served  to  draw  out  those  basic  bits  of  knowledge  which  the 
subject  matter  expert  knr  so  well  that  he  could  not  easily  identify  them 
explicitly.  The  process  of  identifying  the  rules  was  somewhat  tedious,  but 
the  equivalent  prooess  for  a  more  traditional  simulation  system  would  hardly 
be  less  so.  In  the  end,  we  arrived  at  a  good  core  of  production  rules  which 
appear  more  than  adequate  for  the  first  step  in  an  incremental  development. 
The  subject  matter  expert  divided  the  rules  into  two  basic  sets:  those  with 
a  definite  “context,"  and  floating  rules.  As  presented  in  Appendix  D,  the 
rule  structure  appears  rather  constrained  since  only  a  few  rules  appear  to 
bo  potentially  active  at  one  time.  In  part,  this  reflects  constraints  in 
the  basic  ASW  situation:  if  A  is  tr-,.e,  then  actions  B,  C,  D,  and  E  oust 
follow  pretty  much  in  order.  However,  the  floating  rules  are  applicable 


at  any  time,  and  these  rules  will  undoubtedly  break  up  the  rigidity  which 
now  appears  to  be  present  in  the  rule  structure.  Those  familiar  with  production 
system  development  (see,  for  example,  referenoe  McDermott,  1981)  estimate 
that  the  initial  collection  of  production  rules  gathers  from  one- third  to 
one-half  of  the  number  of  rules  present  in  the  finished  system.  We  would 
anticipate  that  the  number  of  floating  rules,  in  particular,  would  increase 
significantly  sr  the  system  develops  and  more  rules  are  added  as  needed. 

A  second  strong  advantage  of  a  production  system  approach  to  modeling 
the  ASWOC  is  the  openness  of  the  control  structure.  Since  the  ASWOC  must 
function  in  an  open-ended  world,  the  ASWOC  model  must  be  adaptabxe  enough 
to  function  in  an  emergent  situation.  This  kind  of  situation  is  handled 
well  by  the  production  system  control  structure.  Production  systems  are 
characterized  by  a  sensitivity  and  a  reactivity  which  arises  from  the  continual 
revaluation  of  the  control  state.  This  is  the  property  which  has  been  sometimes 
described  as  the  “openness"  of  production  rule  based  systems.  This  openness 
is  characterized  by  the  principle  that  "any  rule  can  fire  at  any  time,"  which 
emphasizes  that,  at  any  point  in  the  computation,  any  rule  could  possibly 
be  the  next  one  selected,  depending  on  the  state  of  the  data  base  at  the 
end  of  the  current  cycle.  Compare  this  t.o  a  normal  situation  in  a  traditional 
nrocedural  simulation  where  this,  principle  is  manifestly  untrue:  it  is  simply 
not  the  case  there  that,  depending  on  the  contents  of  the  data  base,  any 
prooedure  in  the  entire  program  could  potentially  be  the  next  one  invoked. 

In  the  next  section,  we  &*■  cribs  in  some  detail  a  specially  tailored 
production  system  to  model  the  rfOC. 
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THE  GENERAL  ARCHITECTURE  OF  THE  ASWOC  MODEL 

In  this  section  we  will  review  the  functional  requirements  for  the  ASWOC 
model,  and  then  describe  a  production  system  based  design  for  meeting  these 
requirements.  Our  attempt  here  is  to  discuss  a  top-level  design;  thus,  many 
issues  of  implementation  are  not  taken  up  at  all.  However,  a  few  key  implemen¬ 
tation  questions  are  discussed  in  the  next  section. 

THE  REQUIREMENTS.  The  primary  functional  requirement  for  the  ASWOC  model 
is  that  the  model  should  be  capable  of  operating  in  the  team  member  environnent 
in  place  of  a  human  ASWOC,  responding  to  situations  a3  they  arise  with  appropriate 
actions  and  suitable  communications.  The  ASWOC  model  constitutes  a  well-defiusd 
subsystem  of  the  full  team  training  system  which  is,  to  some  degree,  self- 
contained.  Nonetheless,  the  ASWOC  model  must  have  the  capability  to  interact 
with  the  external  system,  and  thereby  with  other  team  members  or  other  team 
member  models. 

A  secondary  requirement  of  the  ASWOC  model  is  that  it  support,  the  first 
stage  error  detection  requirements  as  well  as  some  obvious  extensions  to 
traditional  forms  of  performance  measurement.  The  idea  is  that  the  ASWOC 
model,  as  a  nominal  expert  in  the  ASWOC’ s  job,  3hould  provide  the  performance 
measurement  system  with  information  about  several  alternative  correct  sequences 
of  decisions,  actions,  and  oommuni cations  with  measures  of  their  adequacy. 
In  this  capacity,  the  ASWOC  model  must  act  in  a  shadow  role,  and  its  presence 
should  be  invisible  to  the  human  ASWOC. 

Since  the  team  training  system,  as  envisioned,  is  designed  to  develop 
incrementally,  and  since  part  of  the  focus  of  the  system  will  be  it/5  use 
as  a  research  tool,  the  ASWOC  model  is  subject  to  a  third  requirement.  This 
requirement  provides  that  the  ASWOC  model  be  capable  of  supporting  the  validation 
of  performance  measurement,  as  well  as  of  monitoring  its  own  functioning 
and  maintaining  its  own  internal  consistency. 

In  summary,  the  functional  requirements  for  the  ASWOC  model  are  the 
following: 


1  .  The  ASWOC  model  must  be  capable  of  taking  the  place  of  the  human 
ASWOC  in  the  team  training  systems,  making  decisions,  initiating 
actions,  and  producing  communications  as  a  human  might. 

2.  The  ASWOC  model  must  provide  a  standard  of  nominally  correct 
ASWOC  behavior  for  the  performance  measurement  system. 

3.  The  ASWOC  model  must  support  incremental  growth,  modification, 
and  tuning;  moreover,  the  model  should  be  capable  of  supporting 
the  validation  of  performance  measurement. 

THE  OVERALL  CONCEPTION  OF  THE  MODEL.  The  core  of  the  ASWOC  model  shall  be 
a  simple  production  system  consisting  of  a  set  of  rules,  a  data  base,  and 
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a  control  structure.  The  basic  core  would  be  adequate  if  we  were  building 
just  an  ASWOC  expert  model,  but,  by  itself,  this  would  satisfy  only  part 
of  our  first  functional  requirement.  The  complexity  in  the  full  task  of 
meeting  all  the  functional  requirements  calls  for  a  significant  embellishment 
of  the  form  of  a  pure  production  system.  The  basic  ingredients  are  the  same, 
but  there  are  additional  structures  and  even  the  basic  structures  have  additional 
levels  of  complexity. 

Our  design  for  the  ASWOC  model  encompasses  the  following  features: 

1.  A  complete  ASWOC  model  consisting  of  the  following  submodels: 

ASWOC  nominal,  ASWOC  non-standard,  and  ASWOC  performance  measurement. 

2.  A  rule  structure  consisting  of  three  different  kinds  of  rules. 

3.  A  control  structure  capable  of  managing  three  Inferenoe  streams. 

M.  A  single  unified  data  base. 

5.  A  "blackboard"  for  recording  rules  invoked,  state  of  the  world 

inf ortaation,  and  intra-system  messages. 

6.  An  activation  key  limiting  acoess  to  rules  to  the  proper  inference 

stream. 

7.  A  capability  to  identify  an  inference  with  the  inference  stream 

responsible  for  it. 

The  term  "inference  stream"  means  a  sequence  of  applications  of  rules  of 
one  particular  kind. 

At  least  v.wo  of  the  features  of  the  ASWOC  model  run  against  the  spirit 
of  pure  production  systems.  These  are  the  features  which  control  access 
to  rules  (design  feature  #6)  and  which,  so  to  speak,  identify  the  "agent" 
responsible  for  each  inference  (feature  #7).  Both  of  these  items  tend  to 
inhibit  the  autonomy  of  the  rules  and,  to  some  extent,  to  restrict  the  openness 
of  the  rule  control  wtrvoture.  Nonetheless,  the  reasons  for  adopting  these 
features  seem  compelling. 

The  ASWOC  expert— ’that  :.s,  the  standard  model  of  nominally  correct  ASWOC 
behavior — is  represented  by  one  distinguished  subset  of  the  total  set  of 
rules.  Once  each  rule  evaluation  cycle,  the  control  system  evaluates  all 
of  its  own  set  of  rules  (meta-rules  or  strategy  rules,  which  constitute  a 
second  distinguished  set)  to  determine  which  subset  of  the  ASWOC’s  rules 
should  be  potentially  active  during  that  cycle.  The  control  system  then 
evaluates  all  the  applicable  ASWOC  rules,  modifies  the  data  base  for  those 
rules  which  are  invoked,  records  the  inferences  made  on  the  blackboard,  and 
"takes  action"  or  "initiates  communication"  by  making  suitable  entries  on 
the  action-item  area  of  the  blackboard. 
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The  blackboard  has  several  functions  which  will  be  described  in  greater 
detail  later.  Briefly,  the  blackboard  offers  a  summary  of  the  current  state 
of  the  ASUOC  model’s  world  to  mediate  communication  with  the  external  system, 
to  provide  comprehensive  snapshots  of  the  system’s  behavior  for  debugging 
purposes,  and  to  retain  a  complete  history  of  events  to  permit  extensive 
off-line  automated  performance  measurement. 

On-line  performance  measurement  shall  also  be  provided.  A  third  distin¬ 
guished  subset  of  rules  shall  be  performance  measurement  rules,  and  these 
rules  shall  be  capable  of  being  evaluated  onoe  each  rule  evaluation  cycle. 
(We  do  not  offer  a  detailed  design  of  the  performance  measurement  subsystem 
in  this  appendix;  instead,  wo  focus  upon  the  fundamental  representational 
framework  which  shall  support  automated  performance  measurement.)  The  overall 
control  structure  shall  admit  the  possibility  of  different  cycle  lengths 
for  each  kind  of  rule  evaluation  (ASWOC  rules,  strategy  rules,  performance 
measurement  rules)  to  facilitate  an  optimal  mesh  of  the  different  kinds  of 
rule  evaluation. 

The  ASWOC  model  is  capable  of  functioning  in  two  modes.  First,  it  can 
function  in  place  of  the  ASWOC  to  simulate  the  actions  and  decisions  of  a 
human  ASWOC.  Secondly,  it  can  act  as  a  shadow  ASWOC  when  the  human  ASWOC 
is  present.  In  this  case,  the  model  can  make  decisions  and  determine  aotions, 
but  it  will  merely  record  them  and  not  oarry  them  out.  Performance  measurement 
typically  operates  in  this  latter  case,  and  it  is  here  that  the  human  ASWOC' s 
behavior  is  measured  against  the  standard  ASWOC  model  behavior  which  in  turn 
is  derived  from  a  nominal  set  of  rules.  To  validate  performance  measurement, 
it  would  be  desirable  to  exercise  the  performance  measurement  rules  under 
controlled  conditions.  Indeed,  this  is  possible  under  the  first  mode  of 
operation  of  the  ASWOC  model.  The  standard  ASWOC  model  can  operate  to  create 
a  baseline  of  nominal  behavior,  and  an  autonomous  non-standard  ASWOC  model 
could  operate  on  an  augmented,  diminished,  or  modified  set  of  rules.  In 
this  case,  the  performance  measurement  system  would  measure  one  (imperfect) 
non- standard  model  against  the  standard  model  of  nominal  behavior.  This 
approach  allows  for  the  tuning  of  the  performance  measurement  system  under 
circumstances  where  all  variables  are  under  control.  It  also  provides  a 
means  of  identifying  and  correcting  the  deficiencies  in  the  standard  ASWOC 
model,  and  of  reducing  its  unnecessary  rigidities. 

In  the  paragraphs  which  follow,  we  will  discuss  the  rules,  the  control 
structure,  the  data  base  and  the  blackboard  in  greater  detail.  Then  we  will 
give  a  couple  of  examples  of  how  these  system  components  operate  together 
as  a  whole. 

COMPONENTS  OF  THE  SYSTEM.  The  basic  components  of  the  ASWOC  model  are  rules, 
control  structure,  data  base  and  blackboard.  This  section  is  devoted  to 
a  description  of  each  of  these  elements;  it  concludes  with  two  examples  of 
the  model  in  action. 
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Rul es .  Production  rules  make  up  the  heart  of  the  ASWOC  model;  they  represent 
its  core  of  knowledge  and  its  fundamental  atoms.  The  ASWOC  model  contains 
three  distinct  kinds  of  rules.  All  of  these  rules  constitute  a  single  pool 
of  rules  which  can,  in  principle,  be  applied  by  any  "inferencing  agent"  (that 
is,  one  of  the  inference  processes  being  managed  by  the  control  mechanism). 
The  first  rule  type  is  the  basic  condition-action  rule  which  describes  a 
set  of  ASW  conditions  together  with  actions  to  be  taken  when  these  specific 
conditions  oocur.  The  seoond  rule  type  is  a  strategy  rule  or  a  meta-rule 
which  is  used  by  the  controlling  mechanism  to  determine  which  subset  of  ASWOC 
rules  should  be  active  at  any  given  time.  The  third  rule  typ^  is  the  performance 
measurement  rule  which  compares  ASWOC  actions  against  the  comparable  reoonmended 
actions  of  the  standard  ASWOC  model. 

We  shall  not  address  the  precise  format  of  the  rules  here.  Sane  comments 
are  made  about  their  formed  structure  in  the  next  section,  but  here  we  will 
indicate  only  what  their  constituent  elements  are.  Each  rule  shall  consist 
of: 

1 .  An  activation  key.  This  is  a  symbol  which  identifies  the  rule 
to  each  "inferencing  agent"  as  applicable  or  non-applicable. 

2.  A  left-hand-side,  condition,  or  hypothesis.  This  is  a  symbol 
string  which  describes  the  circumstances  under  which  the  rule  can 
be  invoked. 

3.  A  right-hand-side,  action,  or  conclusion.  This  is  a  symbol 
string  which  describes  the  change  to  data  base  which  must  occur 
as  a  result  of  the  application  of  the  rule. 

4.  A  confidence  value.  Thi3  represents  a  relative  judgment  about 
the  appropriateness  of  the  application  of  the  rule.  It  is  intended 
to  provide  a  (situation  specific)  conflict  resolution  parameter 
for  the  control  mechanism,  and  it  should  be  capable  of  being  computed 
dynamically. 

Although  the  rules  all  reside  in  a  single  pool,  it  should  not  be  possible 
for  all  inferencing  agents  to  access  all  the  rules  all  the  time.  For  example, 
the  standard  ASWOC  model  has  no  need  even  to  attempt  to  evaluate  performance 
measurement  rules  since  they  are  simply  not  relevant  to  the  ASWOC' s  job. 
Furthermore,  the  standard  ASWOC  model  has  no  need  even  to  evaluate  the  non¬ 
standard  ASWOC  rules,  since  they  lie  outside  the  boundary  which  defines  standard 
behavior.  The  reason  that  all  the  rules  do  lie  in  a  common  space  is  that 
many  rul.es  ought  to  be  shared  by  the  different  inferenoing  agents.  For  example, 
both  the  non-standard  ASWOC  model  and  the  performance  measurement  model  might 
well  invoke  standard  ASWOC  rules. 

The  activation  key  allows  controlled  access  to  the  rules.  Each  inferencing 
agent  has  a  key  signature  whioh  allows  it  access  to  a  given  subset  of  the 
rules.  The  key  signature  can  be  changed  dynamically  to  enlarge  or  change 
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the  collection  of  accessible  rules  depending  ors  the  state  of  the  environment. 
The  activation  key  offers  a  capability  of  control  which  will  permit  the  infer- 
encing  agents  to  test  appropriate  rules  at  appropriate  times.  The  activation 
key  itself  is  under  the  oontrol  of  strategy  rules  (or  meta-rules),  and'  is 
thus  directly  connected  to  the  system  oontrol  functions. 

The  data  base  which  the  ASWOC  model  uses  is  much  like  the  simple  data 
base  of  a  pure  production  system,  but  there  are  two  notable  differences. 
Although  the  data  base  holds  all  the  symbols  corresponding  to  the  invocation 
of  any  rule  (and  thus  indicates  that  some  inferencing  agent  has  applied  a 
rule  to  produoe  it),  each  symbol  string  generated  by  the  successful  application 
of  the  rule  carries  an  identification  tag  that  indicates  whioh  inference 
process  generated  the  symbol.  For  example,  if  the  non-standard  ASWOC  model 
invokes  the  rule 

"If  a  weapon  is  fired,  the  bridge  should  be  notified  of  the  weapon  firing 
bearing. " 

when  the  data  base  holds  the  string  for  "a  weapon  is  fired"  and  a  tag  whioh 
indicates  that  the  non-standard  ASWOC  model  created  this  symbol  string,  then 
a  new  symbol  string  is  created  in  the  data  base  for  "the  bridge  should  be 
notified  of  the  weapon  firing  bearing",  and  this  symbol  string  is  also  tagged 
with  the  non-standard  ASWOC*  s  identification  mark.  This  allows  the  non-standard 
model  to  proceed  relatively  independently  of  the  standard  model  in  generating 
its  own  strings  of  inferenoes  without  affeoting  the  funotions  of  the  nominal 
ASWOC  model. 

Control  Structure.  The  basic  funotions  of  the  control  structure  shall  be 
to  direot  the  application  of  the  production  rules  and  to  maintain  the  interfaoe 
with  the  external  system.  The  oontrol  structure  shall  be  capable  of  maintaining 
three  distinct  inference  streams  and,  thereby,  of  coordinating  three  different 
"agents"  who  can  apply  rules.  (The  idea  of  quasi- independent  "agents"  who 
oan  aot  by  choosing  rules,  applying  them,  and  modifying  their  memories  corres¬ 
pondingly  offers  a  useful  way  to  think  and  talk  about  this  multifaceted  control 
structure.) 

Each  basic  rule  control  cycle  for  each  agent  can  be  broken  down  into 
two  phases:  recognition  and  action.  The  recognition  phase  can  be  further 
subdivided  into  selection  and  conflict  resolution  operations.  In  the  selection 
process,  one  or  more  potentially  applicable  rules  are  chosen  from  the  basic 
pool  of  rules,  and  are  then  passed  on  to  the  conflict  resolution  algorithm, 
which  ohooses  one  of  them. 

The  ohoioe  which  occurs  at  this  selection  phase  is  fundamental.  At 
this  stage,  the  oontrol  mechanism  operates  a  symbol  scanner  which  scans  the 
left-hand-sides  of  all  rules  applicable  for  a  given  agent  at  that  time,  and 
collects  all  rules  which  evaluate  successfully.  Then  rules  whose  confidence 
values  lie  below  a  certain  threshold  value  are  rejected,  and  the  remaining 
rules  are  passed  on  to  the  conflict  resolution  algorithm. 
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The  conflict  resolution  algorithm  must  decide  which  of  the  applicable 
rules  will  actually  be  invoked.  The  purpose  of  conflict  resolution  is  to 
see  that  the  actions  taken  by  the  model  have  a  well-defined  direction  toward 
an  overall  goal;  if  rules  were  to  be  invoked  whenever  they  were  applicable, 
the  agent's  "attention"  might  well  wander  from  one  direction  or  activity 
to  another  without  focus.  The  overall  goal  and  subsidiary  subgoals  are  embodied 
in  two  ways.  First,  as  confidence  values  attached  to  the  rules:  this  gives 
a  means  of  rendering  one  applicable  rule  more  desirable  than  another.  Secondly, 
as  an  algorithm  to  generate  confidence  values  automatically:  this  provides 
a  means  of  shifting  smoothly  from  one  subgoal  to  another  as  the  situation 
evolves. 

When  it  is  finished,  the  conflict  resolution  algorithm  recommends  one 
or  more  rules  to  apply.  Then  the  action  phase  begins.  The  chosen  rules 
are  applied,  the  data  base  is  modified  based  on  the  right-hand-side  of  the 
rules  invoked,  and  the  oontents  of  the  blackboard  are  updated. 

Control  of  the  blackboard  is  a  significant  aspect  of  the  control  structure's 
responsibilities  inasmuoh  as  the  blackboard  maintains  all  the  communication 
links  with  the  external  system.  From  one  section  of  the  blackboard,  the 
control  mechanism  can  extract  new  state-of- the-world  information  as  well 
as  messages  directed  to  the  ASWOC  by  other  team  members  or  by  other  simulated 
personnel.  On  another  section  of  the  blackboard,  the  oontrol  mechanism  can 
record  messages  or  commands  directed  toward  other  parts  of  the  system. 

Data  Base.  The  data  base  for  the  ASWOC  model  shall  consist  Of  two  parts. 
The  largest  part  shall  be  comprised  of  an  area  of  working  memory  where  the 
symbol  strings  generated  by  the  invocation  of  rules  shall  be  recorded.  Peri¬ 
odically,  the  working  memory  will  be  cleaned  up  by  a  garbage  collection  algorithm 
whose  function  is  to  restore  and  re-allocate  memory  space  which  is  holding 
expired  symbol  strings. 

The  second  part  of  the  data  base  shall  consist  of  one  panel  of  the  blackboard 
where  state-of- the-world  information  is  stored.  Both  new  messages  and  current 
environmental  data  are  stored  in  this  area  of  the  blackboard.  Information 
contained  in  the  messages  is  eventually  moved  to  working  memory,  but  current 
information  about  the  features  of  the  world  which  the  ASWOC  oannot  change 
directly  (e.g.,  submarine  position,  ownship  position,  sonar  data,  etc.)  shall 
always  remain  recorded  in  this  panel  of  the  blackboard.  The  control  mechanism 
shall  maintain  the  blackboard  internally;  a  system  utility  routine  shall 
control  the  external  system's  use  of  the  blackboard  (i.e.,  reading  from  or 
writing  to) . 

The  Blackboard.  The  notion  of  an  abstract  blackboard  as  a  software  device 
for  recording  system  information  is  probably  due  to  the  developers  of  the 
HEARSAY-II  speech  understanding  system  at  Carnegie-Mellon  University  (Erman 
and  Lesser,  1980).  Originally,  the  blackboard  was  supposed  to  record  hypotheses 
made  by  the  various  knowledge-source  components  of  the  system  as  an  unknown 
utterance  was  being  analyzed.  In  this  way,  separate  elements  of  a  large 
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syrtem  could  "share  their  ideas"  at  the  same  time  that  their  interactions 
were  being  kept  to  a  minimum.  In  our  case,  the  blackboard  is  more  of  a  data 
tablet,  but  it  also  functions  very  much  as  a  communication  device.  It  provides 
one  panel  for  communications  received  from  the  external  system,  and  one  panel 
for  messages  or  ooomands  directed  to  the  external  system.  Another  panel 
records  the  inferences  and  actions  taken  (or  recommended)  by  the  nominal 
ASWOC  model.  A  fourth  panel  either  records  the  human  ASWOC’s  activities 
(if  the  system  is  operating  in  this  mode),  or,  in  another  mode,  records  the 
inferences  made  and  actions  taken  by  the  non-standard  ASWOC  model.  A  fifth 
panel  records  the  invocation  of  performance  measurement  rules.  Table  1-1 
summarizes  these  blackboard  features. 


TABLE  I- 1.  THE  BLACKBOARD  FEATURES. 


Panel  1 .  'rhis  panel  holds  state-of-the-world  information  as 
ell  as  new  messages  which  have  just  been  reoeived 
from  the  external  system. 


Panel  2.  This  panel  holds  a  record  of  inferences  made  by  the 
nominal  ASWOC  model  and  actions  which  are  taken  or 
reoommended  by  the  model. 


Panel  3.  This  panel  holds  either  a  record  of  the  human  ASWOC* s 
activities  or  a  record  of  the  non-standard  ASWOC* s 
Inferences  made  and  actions  taken. 


Panel  4.  This  panel  maintains  a  record  of  the  performance  measurement 
rules  which  were  invoked. 


Panel  5.  This  panel  holds  a  record  of  messages  transmitted  to 
the  external  system  as  well  as  commands  issued  by  the 
ASWOC  model. 


Besides  providing  a  necessary  medium  for  intra-system  oommuni cation, 
the  blackboard  serves  at  least  two  other  significant  functions.  The  first 
of  these  is  as  a  debugging  aid.  To  disentangle  the  several  inference  streams 
operating,  and  to  understand  complex  system  interactions,  the  system  can 
simply  take  periodic  snapshots  of  the  blackboard  and  store  them  on  disk  for 
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off-line  analysis.  This  would  allow  the  software  engineer  to  reconstruct 
complicated  situations,  follow  out  the  inference  streams,  and  traoe  commun¬ 
ications  to  and  from  the  ASWOC  model. 

The  third  function  of  the  blackboard  is  to  support  off-line  automated 
performance  measurement.  Since  both  human  ASWOC  aotions  and  recommendations 
of  the  nominal  ASWOC  model  are  recorded  on  the  blackboard,  a  sequenoe  of 
snapshots  of  the  blackboard  would  provide  a  substantial  exeroise  summary 
upon  which  many  varieties  of  measures  of  performance  might  be  tested. 

Examples.  The  full  enhanced  production  system  design  which  we  have  described 
is  relatively  complicated,  and  we  have  really  only  sketohed  the  abstract 
skeleton  of  a  system  which  will  be  capable  of  many  kinds  of  "intelligent" 
(or  intelligent-seeming)  behavior.  To  give  a  better  idea,  or  a  better  feel, 
of  how  the  system  will  eventually  function,  let  us  consider  two  simple  examples 
which  will  give  a  flavor  of  the  dynamic  character  of  the  ASWOC  model. 

For  the  first  example,  let  us  consider  a  situation  where  the  human  ASWOC 
is  present  and  the  ASWOC  model  is  functioning  as  a  nominal  standard  of  behavior. 
In  a  very  simplified  way,  we  will  set  up  the  ourrent  state  of  the  system: 
rules,  blackboard  status,  and  data  base. 

Let  us  assume  that  the  ASW  team  has  localized  the  position  of  a  hostile 
submarine,  has  put  the  snip  in  a  position  to  fire  a  weapon  at  the  submarine, 
has  cleared  friendly  tracks  from  the  weapon  danger  area,  and  has  fired  a 
torpedo  at  the  submarine.  The  human  ASWOC  is  in  control  of  the  situation, 
and  the  ASWOC  model  is  acting  in  its  shadow  oapacity.  The  ASWOC  model  will 
determine  a  oourse  of  action  to  follow,  but  the  human  ASWOC  makes  his  own 
deoisions  and  the  ASWOC  model  must  record  its  own  recommendations  and  follow 
along  behind  the  human  ASWOC.  The  complete  ASWOC  model  has  a  large  data 
base,  a  significant  set  of  rules,  and  a  large  blackboard  area.  But,  to  make 
the  example  manageable,  we  will  Just  look  at  little  pieces  of  the  big  structures. 

Assume  that  the  ASWOC  model  data  base  consists  of  the  following  strings: 

Torpedo-has-been- fired-at- submarine 

Firing-bearing- is- XXX 

Weapon-danger-area-has-been-cleared 

Suppose  that  the  collection  of  ASWOC  rules  includes  these  rules  which 
are  active  at  the  present  time: 

PI.  If  a  weapon  has  been  fired  at  the  submarine,  then  the  SAU  oommander 

should  be  notified  of  the  firing  bearing  (ANA3,  0,90). 

P2 .  If  a  weapon  has  been  fired  a.  the  submarine,  then  ownship  should 

retire  to  a  safe  distance  (ANA3,  0.95). 
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P3.  If  ownship  is  to  retire  to  a  safe  distance  ana  the  submarine  is 
off  the  starboard  bow,  then  ownship  must  turn  90  degrees  to  port  and 
aooelerate  to  20  knots  (ANA 2,  0.92). 

PA.  If  ownship  is  to  retire  to  a  safe  distance  and  the  submarine  is 
off  the  port  bow,  then  ownship  must  turn  90  degrees  to  starboard  and 
aooelerate  to  20  knots  (ANA2 ,  0.92). 

PS.  If  ownship  is  to  turn  or  change  speed,  then  the  bridge  must  be  notified 
of  desired  course  and  speed  (ANA1,  0.95). 

Here  the  rules  are  in  the  usual  ocndition-action  format  and  two  additional 
elements  are  included.  These  are:  the  activation  key  describing  who  has 
access  to  the  rule  and  when,  and  a  confidence  value  which  is  a  dynamic  measure 
of  the  priority  of  the  rule.  (We  assume  here  that  this  oonfidenoe  value 
is  a  kind  of  subjective  probability  measure  with  a  value  between  0  and  1 
and  with  larger  values  indioating  higher  priorities.) 

Furthermore,  we  assume  that  the  blackboard  contains  the  following  infor- 
mation: 

Submarine  is  at  range  RRRR  yards  and  bearing  in  degrees 
Submarine  speed  is  SS 
Ownship  course  is  CCC 
Ownship  speed  is  S 

Sonar  oontact  with  submarine  is  strong 

In  addition,  there  are  performance  measurement  rules  which  are  aotive. 
Assume,  for  simplicity,  that  there  are  only  two  involved  here. 

Ml  .  If  a  weapon  is  f'ired,  then  the  ASWOC  must  notify  the  SAU  commander 
of  the  oorrect  firing  bearing  (PM3,0,80). 

M2.  If  a  weapon  is  fired,  then  the  ASWOC  must  command  a  turn  and  a  change 
of  speed  which  moves  ownship  to  a  safe  area  (PM3,0.90). 

Now  we  turn  the  system  on,  and  watch  the  situation  evolve  dynamically. 
Suppose  that  the  ASWOC  model  starts  with  an  activation  key  value  ANA*  (so 
that  any  rule  with  activation  key  value  beginning  ANA  is  applicable).  The 
control  meohanism  evaluates  the  applicable  rules.  (All  the  rules  listed 
are  potentially  applicable  since  their  activation  keys  all  match  thu  ASWOC 
model's  own.)  The  rule  interpreter  determines  that  rules  PI  and  P2  tre  satis¬ 
fied.  The  oonflict  resolution  algorithm  determines  that  rule  P2  has  the 
higher  priority,  and  thus  it  is  activated.  This  rule's  activation  key  ic 
then  changed  to  ANO  to  avoid  having  it  evaluated  again  on  the  next  cycle. 
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The  string  "ownship-should-retire-to-a-saf e-distanoe"  is  written  to  the  data 
base.  Meanwhile,  the  blackboard  is  updated  with  new  values  of  the  submarine’s 
range,  bearing,  and  speed,  as  well  as  ownship  course  and  speed  and  the  status 
of  sonar. 

On  the  next  cycle,  the  applicable  rules  are  again  evaluated.  This  time 
rule  PI  is  still  applicable,  and  rules  P3  and  P4  have  become  applicable. 
The  oonflict  resolution  algorithm  chooses  rule  P3  or  P4  contingent  on  the 
position  of  the  submarine.  (This  really  requires  another  rule— or  an  addition 
to  P3  and  P4— to  determine  whether  the  submarine  is  to  the  port  or  the  starboard 
side  depending  on  the  blackboard  information.)  The  ASWOC  model  then  recommends 
a  turn  and  a  ohange  of  speed.  (This  recommendation  takes  the  form  of  a  message 
on  the  blackboard  intended  for  the  bridge.)  Sinoe  the  human  ASWOC  is  present, 
all  the  ASWOC  model’s  "actions*'  will  be  recommendations  for  action,  and  will 
be  recorded  for  use  by  the  performance  measurement  rules. 

When  rule  P3  or  ?4  is  applied,  the  activation  keys  of  both  rules  are 
reset.  The  blackboard  is  updated  with  new  position  and  speed  Information 
for  ownship  and  the  submarine.  Furthermore,  the  data  base  is  updated  to 
add  the  string  ’’ship-is~to-turn-90-degrees-to-starboard(port)-and-ttooelerate- 
to-  20-knots",, 

On  the  next  cycle,  rules  PI  and  Pi5  are  found  to  be  applicable,  and  rule 
P5  has  greater  priority.  Thus  it  is  invoked,  and  a  message  is  written  to 
panel  5  of  the  blackboard  directing  the  bridge  to  turn  the  ship  90  degrees 
to  port  (starboard)  and  accelerate  to  20  knots. 

Finally,  on  the  next  oycle  rule  PI  is  applied.  This  causes  another 
message  to  be  written  to  panel  5  of  the  blackboard.  The  message  this  time 
is  direoted  to  the  SAU  commander  and  it  informs  him  of  the  weapon  firing 
bearing. 

So  far,  we  have  described  only  the  recommendations  which  the  AS\  X  reoords 
as  the  human  ASWOC  is  actually  responding  to  the  situation.  It  is  possible, 
for  example,  that  the  human  ASWOC  chooses  to  Inform  the  SAU  commander  of 
the  weapon  firing  bearing  before  ordering  ownship  to  proceed  to  a  safe  area. 
In  this  case,  the  ASWOC  model  should  record  its  own  choice  oh  panel  2  of 
the  blackboard,  but  should  then  continue  to  follow  along  with  the  human  by 
noting  that  the  rule  PI  has  been  invoked,  and  basing  its  further  actions 
upon  that. 

The  performance  measurement  rules  Ml  and  M2  indicate  that  getting  the 
ship  to  a  safe  plaoe  has  priority  over  speaking  with  the  SAU  commander,  and 
it  would  note  in  panel  4  of  the  blackboard  that  the  human  reversed  this  priority. 

A  weak  human  ASWOC  might  need  help  deciding  what  to  do  after  firing 
a  weapon  at  the  submarine.  In  this  event,  the  ASWOC  model  is  capable  of 
using  the  rules  at  its  disposal  to  construct  a  prompting  message  like:  "Since 
a  weapon  has  been  fired,  the  ship  should  be  moved  to  a  safe  place," 
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Our  second  example  starts  with  the  same  setting  as  the  first  example, 
except  that  the  mode  of  operation  of  the  system  is  different.  This  time 
we  will  assume  that  there  is  no  human  ASVOC  present,  and  that  the  nominal 
ASWOC  model  is  aoting  as  the  ASWOC  in  the  team.  Let  us  suppose  that  we  are 
trying  to  verify  that  the  performance  measurement  rules  Ml  and  M2  are  applied 
correctly,  so  we  will  set  up  a  non-standard  ASWOC  model  with  access  to  rules 
PI  through  P6  plus  the  additional  rules 

P7.  If  a  weapon  has  been  fired  at  tha  submarine,  then  the  weapon  should 
be  followed  in  to  the  submarine  to  put  ownship  in  position  to  fire  a 
second  weapon  (ANS3,0,98). 

With  this  rule  present,  the  non-standard  ASWOC  model  will  apply  it  at  once 
in  preference  to  PI  and  P2  (since  its  priority  is  higher)  and  will  not  retreat 
to  a  safe  distance.  The  nominal  ASWOC  model,  however,  will  invoke  rules 
P2,  P3  (P4).  P5,  and  PI  and  thereby  recommend  getting  to  a  safe  area  while 
notifying  the  SAU  commander  of  the  weapon  firing  bearing.  The  performance 
measurement  system  then  notes  that,  although  a  weapon  has  been  fired,  the 
ship  has  not  moved  to  a  safe  area. 

SOME  COMMENTS  ABOUT  IMPLEMENTATION 

The  ASWOC  model  is  a  oomplex  entity  which  consists  of  several  distinct 
but  interacting  parts.  To  go  from  the  design  outline  whiob  we  have  presented 
here  to  a  oomplete  ASWOC  subsystem,  one  must  prooeed  deliberately.  In  this 
section,  we  will  sketoh  an  implementation  plan  which  focuses  on  developing 
ar.  expert  ASWOC  model  as  an  initial  goal  with  other  system  features  added 
incrementally.  This  approach  should  produoe  a  working  system  fairly  early 
in  the  process  so  that  a  subject  matter  expert  can  evaluate  the  models  expertise, 
modify  rules  as  necessary,  and  recommend  new  rules  as  gaps  in  the  model's 
knowledge  become  apparent. 

Our  emphasis  in  this  implementation  plan  is  on  incremental  development. 
At  this  stage  in  the  development  of  expert  system  technologies,  extra  effort 
should  be  made  to  evaluate  the  system  as  it  grows  to  spot  potential  problems 
early.  The  design  we  have  presented  offers  a  good  opportunity  to  use  the 
advantages  of  incremental  development  in  both  technical  and  management  contexts. 
Technically,  an  incremental  approach  is  likely  to  provide  a  smoother  implemen¬ 
tation  with  clearer  milestones  and  more  definite  baseline  subsystems.  From 
the  manager's  perspective,  the  incremental  approach  should  provide  more  obvious 
and  more  frequent  points  for  management  review  of  the  developing  product. 

The  first  stage  in  the  development  of  the  ASWOC  subsystem  is  to  implement 
an  expert  model  of  the  ASWOC  independent  of  the  external  team  training  system. 
Communications  with  the  model  at  this  stage  would  be  done  by  keyboard  entry; 
this  would  also  serve  to  simulate  communications  with  other  members  of  the 
full  ASW  team.  In  this  way,  the  early  focus  would  be  upon  building,  testing, 
evaluating,  and  correcting  a  basic  model  of  the  ASWOC' 3  operational  knowledge. 
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Because  this  knowledge  is  embodied  in  self-contained  rules,  a  very  limited 
ASWOC  model  oould  be  brought  up  first  with  a  small  set  of  rules,  and  then 
new  rules  could  be  added  slowly  to  broaden  the  knowledge  base. 

The  initial  development  will  require  a  preliminary  control  structure, 
an  early  form  of  the  blackboard,  the  data  base,  and  a  basic  core  of  ASWOC 
rules.  Very  early  in  the  process,  there  will  be  a  need  to  decide  upon  a 
format  for  the  maohine  representation  of  the  rules,  as  well  as  a  need  for 
an  auxiliary  program  to  translate  English  language  forms  of  the  rules  into 
internal  machine  format,  and  vice-versa.  Our  recommendation  at  this  point 
is  that  the  rules  be  represented  as  an  ordered  four- tuple  of  symbols: 


(K,L,R,C) 


where  K  is  the  activation  key,  L  is  the  lef t-hand-3ide  (condition)  of  the 
rule,  R  is  the  right-hand-side  (action)  of  the  rule,  and  C  is  the  confidence 
value  or  priority  of  the  rule. 

Once  a  basic  ASWOC  model  is  running,  the  subjeot  matter  expert  can  work 
with  the  model  to  evaluate  its  judgments  and  recommend  new  rules.  At  this 
stage,  the  standard  or  nominal  ASWOC  model  should  be  regarded  as  tentatively 
complete,  and  that  should  be  the  first  baseline  for  the  system. 

The  next  phase  of  development  should  be  the  expansion  of  the  basic  control 
structure  to  permit  maintenance  of  three  different  inference  streams  operating 
over  three  potentially  different  cycle  lengths.  During  this  phase,  all  of 
the  features  of  the  blackboard  should  be  implemented  and  tested  as  much  as 
possible. 

The  third  phase  of  development  should  be  the  inclusion  of  the  standard 
ASWOC  model  into  the  broader  context  of  the  more  sophisticated  control  structure. 
At  the  same  time,  the  capabilities  for  the  parallel  operation  of  a  non-standard 
ASWOC  model  should  be  checked  out.  Sinoe  all  of  the  prerequisite  structures 
(blackboard,  control  structure,  rule  base)  have  been  prepared  by  this  stage, 
the  non-standard  ASWOC  model  should  be  functional,  ard  its  activities  could 
be  studied  now  by  injecting  rules  unique  to  it. 

The  fourth  stage  of  development  should  be  the  inclusion  of  performance 
measurement  rules.  By  now,  the  control  structure  already  has  the  capability 
to  engage  the  performance  measurement  rules  and  record  the  results  on  the 
blackboard.  It  is  anticipated  that  there  will  be  a  significant  amount  of 
test  and  evaluation  performed  at  this  point  during  which  the  operation  of 
the  performance  measurement  rules  would  be  scrutinized,  the  standard  ASWOC 
rules  tuned,  and  the  coordination  activities  of  the  control  structure  extensively 
checked  out. 

The  final  stage  of  development  should  be  the  integration  of  the  ASWOC 
subsystem  with  the  rest  of  the  team  training  system. 
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CONCLUSIONS 

In  this  appendix  we  have  presented  an  outline  of  a  design  for  a  knowledge- 
based  ASWOC  model  which  is  intended  to  function  as  a  subsystem  of  the  full 
team  training  demonstration  system.  The  fundamental  structure  of  the  ASWOC 
m/vUti  is  baaed  on  an  au^Mnted  production  rule  system  wherein  the  basic  components 
of  a  simple  production  system  are  supplemented  with  additional  structures. 
Our  intent  in  this  design  has  been  to  show  that  a  production  system  provides 
a  sound  basis  for  the  simulation  of  a  human  ASWOC’ s  activities,  and  that 
the  production  rule  based  approaob  may  well  be  superior  to  traditional  procedural 
approaches.  Our  design  should  indicate  at  least  the  possibilities  which 
are  inherent  in  a  knowledge-based  simulation  of  the  ASWOC. 
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APPENDIX  J 

ANALYSIS  OF  COMMUNICATIONS  IN  THE  TRAINING  ENVIRONMENT 

Vhile  the  importance  of  good  oomrauni cations  in  the  CIC  environment  is 
widely  appreciated,  &he  design  of  a  training  system  requires  specific  information 
about  bow  communications  affect  mission  outcome.  Access  to  ASW  training 
tapes  was  gained  through  the  Navy  Personnel  Research  and  Development  Center 
(NPRDC)  Can  Diego.  From  these  tapes,  three  exercises  which  represented  good, 
average,  and  poor  overall  performance  of  the  same  type  training  exercise 
wire  selected  and  transcribed.  The  intent  was  to  determine  if  mission  success 
was  related  to  communications.  It  was  found  that  in  fact,  the  ship  that 
had  a  smooth  exchange  cf  vital  information  had  far  better  success  than  did 
t  ship  that  did  not  have  6.  smooth  exchange.  More  importantly,  the  analysis 
revealed  that  the  mission  outcome  was  indued  reflected  in  the  communications. 

To  oonduot  the  analysis  the  communications  weza  broken  down  by  circuit. 
Ship  1,  Judged  to  be  good,  attacked  the  submarine,  and  all  stations  were 
involved  as  a  team,  keeping  each  other  advised  of  all  pertinent  information. 
Ship  2,  judged  to  be  average,  attacked  the  submarine  and  was  toerefore  successful 
in  their  mission,  but  many  times  all  stations  were  not  aware  of  wbat  was 
going  on  or  the  evaluator’s  intention.  This  was  because  of  a  communications 
breakdown  between  stations.  Ship  Bs  judged  to  be  poor,  had  very  little  team 
interaction.  In  fact,  when  UR  did  make  an  attempt  to  communicate,  there 
was  no  response  from  the  other  stations.  In  this  case  the  ship  was  not  successful 
j.n  attacking  the  submarine  and  was  attacked.  These  communication  exchanges 
or  lack  cf  exchanges  will  become  obvious  as  one  reads  ohe  communication  trans¬ 
cript. 

A  key  to  the  transcriptions  which  follow  is  provided  in  Table  B- 1 . 
In  the  following  paragraphs  some  specific  details  of  the  transcriptions  are 
discussed. 

GOOD  COMMUNICATIONS  -  SHIP  1 

As  the  transcription  in  Table  B-<°  shows,  there  was  a  running  dialogue 
on  Ship  1  between  the  evaluator  and  UB  who  acted  as  the  filter  for  sonar. 
The  evaluator  kept  UB  advised  of  all  contacts,  intentions,  and  accepted  connun- 
ieat.lona  to  gain  or  hold  contact,  UB  kept  the  evaluator  informed  of  the 
contact’s  estimated  position,  probable  course,  and  speed.  Ship  1  displayed 
excellent  team  behavior  in  the  constant  comparison  made  between  CIC  and  UB 
to  arrive  at  the  best  solution.  Below  are  some  of  the  internal  and  external 
communications  and  comments  on  correctness. 

Example  1 : 


From  CIC  to  UB  ’'Another  MAD  man  262  10900  L?  weapon  away  safe  time 
23.” 
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From  UB  to  CIC:  "That *3  a  pretty  radical  jump.  I  don’t  think  he  can 
make  that." 

From  CIC  to  OB:  "Yeah,  I  hold  him  022  14  Kts." 

This  good  exchange  ensured  that  both  UB  and  CIC  shared  the  same  estimated 
position  of  the  submarine. 

Example  2: 

From  CIC  to  UB:  "Do  you  have  an  estimated  position  on  him?” 

From  UB  to  CIC:  Roger,  26  8  8500  should  be  coming  into  sonar  range  in 
a  minute  now." 

From  CIC  to  UB:  "ASAC  is  holding  about  the  same." 

Here  the  evaluator  not  only  compared  the  information  from  UB  and  CIC, 
but  from  the  aircraft  as  well.  This  was  very  good  team  behavior. 

Example  3: 

From  RN  to  CR:  "Contact’s  estimated  position  272  6800  from  RN." 

Frequent  communications  like  this  kept  the  other  ship  of  the  SAU  advised 
of  the  action  and  directed  his  movements. 

Keeping  the  assist  ship  advised  of  even  estimated  positions  is  always 
a  good  idea  so  that  the  assist  ship  can  position  himself  to  assist  or  detect 
the  contact. 

Example  4: 

From  RN  to  CR:  "Scramming  L2 ,  H3  to  the  north.  Assume  brother  if  you 
are  hot." 

The  assist  ship  should  have  assumed  the  duties  as  attacking  3hip  when 
she  got  contact.  In  this  communication,  the  training  ship  demonstrated  good 
control  and  team  work  by  notifying  the  assist  ship  of  the  helicopter's  movement 
and  giving  the  assist  ship  the  opportunity  to  become  the  attacking  ship. 
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TABLE  J-1.  ABBREVIATIONS,  ACRONYMS,  AND  PROSIGNS. 
The  following  abbreviations  are  used  in  Tables  J-2,  J-3,  and  J-4. 


Abbreviations 

Meaning 

A/C 

Aircraft 

ANS 

One-half  of  a  whole  number,  e.g. ,  1500  =  1  ANS 

AE 

Out 

AS 

Form  a  surfaoe  attack  unit  (SAU)  (when  appearing 
text) 

in  the 

AS 

All  stations  (when  in  the  TO  -  FM  column 

A/S 

Assist  ship 

ASROC  WHITE 

Ships  individual  ASW  condition  meaning  Active  Beam 
Search 

to  Beam 

AX 

Call  sign  of  the  officer  in  tactical  command 

BF 

Call  sign  of  the  protected  unit 

BNG 

Bearing 

BE 

Bridge 

BT 

Break  (new  subject) 

CIC 

Combat  Information  Center 

COR PEN  J 

Type  of  turn 

C8 

Call  sign  of  the  assist  ship 

C/3 

Call  sign 

Dcsig 

Designate 

DR 

Call  sign  of  exercise  ship  2 

ETA 

Estimated  time  of  arrival 

Form  Y 

Type  of  formation 

H3 

Dipping  helo  call  sign 

IMX 

Immediate  execute 

ISA 

I  say  again 

K  SP 

Indicates  a  speed  range,  e.g,,  18  -  22 

K 

Over 

KTS 

Kr;ots 

LS 

Call  sign  of  the  LAMPS  helo 

LY 

Call  sign  of  exercise  ship  3 

M  speed 

MY  speed 

M  Cor per 

MY  course 
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TABLE  J-1.  ABBREVIATIONS,  ACRONYMS,  AND  PROSIGNS,  CONT. 


PK 

RAR 

RN 

RNG 

SAC 

SAU 

SON 

SP 

STBY 

TDA 

TSR 

TURN  Z 


Computer  estimated  position  of  the  oontact 
Roger  out 

Call  sign  of  exercise  ship  1 
Range 

Scene  of  action  commander 

Surface  attack  unit 

Sonar 

Speed 

Standby 

Torpedo  danger  area 
Tactical  sonar  range 

Indicates  zigzag  plan  (when  followed  by  a  desig  and  letter 
to  indicate  which  zigzag  plan) 

Underwater  battery 

Execute 

Execute  to  follow 

Tack  (used  to  separate  signals  to  avoid  confusion) 
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AVERAGE  COMMUNICATIONS  -  SHIP  2 

The  transcription  in  Table  J-3  reveals  that  the  CIC  in  Ship  2  did  not 
support  UB  and  Sonar  with  all  the  information  they  had.  The  evaluator  appeared 
to  want  all  the  known  information  about  the  submarine  for  himself  and  did 
not  want  to  share  it  with  UB  and  Sonar. 

Example  1 : 

From  CR  to  DR:  "Lance  contact  BNG  262-71." 

This  report  from  the  assist  ship  never  reached  UB  or  ,;onar.  This  information 
should  have  been  conveyed  to  all  stations  to  assist  them  in  gaining  contact. 

Example  2: 

From  DR  to  CR:  "I  have  MAD.  I  have  dipper.  Revise  possub  classification 
high. " 

The  reclassification  was  fine  but  the  position  of  the  submarine  was 
not  reported  to  the  assist  ship  or  to  UB  and  Sonar.  The  assist  ship  oould 
not  correlate  this  contact  to  its  own,  and  Sonar  did  not  know  where  to  concentrate 
the  sonar  sear oh „ 

Example  3: 

From  SON  to  CIC:  "Request  search  arcs." 

From  CIC  to  SON:  "Negative  at  this  time." 

The  evaluator  in  this  case  had  the  best  information  available  and  should 
have  automatically  assigned  new  searoh  arcs. 
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POOR  COMMUNICATIONS  -  SHIP  3 

The  transcription  in  Table  J-H  shows  that  in  ship  3»  CIC  and  UB  appeared 
at  times  to  be  competing  with  each  other  rather  than  operating  as  a  team. 
UB  attempted  to  give  the  submarine  position  to  CIC,  but  CIC  made  no  attempt 
to  verify  or  correlate  the  position  and  report  to  UB.  This  team  interrupted 
orucial  reports,  talked  at  the  same  time,  and  in  general  did  not  communicate 
effectively  with  each  other.  This  resulted  in  mission  failure. 

Example  1 : 

From  UB  to  CIC:  "Is  the  aircraft  hot?" 

From  CIC  to  UB:  "Yes  we  have  an  approximate  course  on  the  submarine." 

When  the  submarine's  position,  course,  or  speed  is  known,  a1!  stations 
and  ASW  units  should  be  notified  automatically.  This  allows  everyone  to 
concentrate  on  that  position  and  to  set  up  a  dead  reckoning  track. 

Example  2: 

From  UB  to  CIC:  "Target's  course  and  speed?" 

CIC  had  received  the  course  and  speed  from  the  assist  ship,  but  did 
not  respond  to  this  request  from  UB. 

Example  3: 

From  SON  to  CIC:  "Sonar  oontact..." 

This  most  critical  report  was  interrupted  by  CIC. 

Example 

From  CIC  to  BR:  "Bridge  combat  bridge  combat  turn..." 

This  transmission  was  also  interrupted.  Sonar  was  attempting  to  report 
a  sonar  contact  when  CIC  interrupted  them  to  make  a  maneuver  and  TJB  interrupted 
CIC  to  report  a  plotter  malfunction.  This  entire  sr-ies  of  communications 
was  unsatisfactory  and  showed  poor  team  behavior. 

Example  5: 

From  UB  to  CIC:  "I  am  going  to  put  a  bloodhound  in  the  water." 

From  SON:  "Classification  possible  torpedo." 

From  UB  to  All  Stations:  "Standby." 
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From  CIC:  "OK;  welt  wait." 

From  OB  to  CIC:  "Firing  bearing  180." 

This  was  another  aeries  of  unsatisfactory  oommuni cations,*  First  the 
evaluator  in  CIC  should  order  weapons  fired,  not  OB.  The  stage  was  set  for 
this  earlier  when  OB  was  allowed  to  prep  a  weapon  without  being  ordered  to 
do  so,  and  the  evaluator  lost  oontrol.  Next  OB,  Sonar,  and  CIC  were  only 
getting  bits  and  pieoes  of  information  out,  not  full  reports.  Finally,  the 
evaluator  in  CIC  had  a  habit  of  beginning  each  transmission  with  "O.K.* 
Therefore,  when  OB  gave  the  "Standby,"  CIC  said,  "O.K. ,"  which  OB  took  as 
permission  to  fire.  As  soon  as  CIC  said  "O.E.,"  OB  fired.  What  CIC  was 
attempting  to  say  was  "wait,"  but  it  oame  too  late. 

The  external  communications  for  Ship  3  were  not  good  either.  Because 
of  the  overall  poor  performance,  they  never  really  had  a  contact  to  report. 
The  first  error  made  was  early  in  the  exercise  when  the  sinker  was  first 
reported. 

Example  6: 

From  LY  to  CR:  "Datum  A1  260-12  time  0902  Source  L2,  classify  possub 
high. " 

L2  is  the  LAMPS  helicopter  and  was  the  only  one  to  report  a  radar  sinker. 
This  is  not  enough  for  a  classification  of  possub  high. 

Example  7: 

From  LY  to  CR:  "H3  weapon  away  safe  time  0922  water  entry  point  will 
follow. " 

This  report  was  correct,  but  it  was  the  first  indication  that  the  Dipping 
Helo  was  In  contact,  and  even  this  was  not  reported  internally. 
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QUALITY  VS.  QUANTITY  OF  TRANSMISSIONS 

To  summarize  t!ie  analysis,  it  vas  found  that  tbe  ship  with  the  best 
exchange  of  necessary  information  did  the  best  job.  It  is  important  here 
to  stress  the  quality  of  tbe  transmissions,  not  quantity.  As  demonstrated 
in  Tables  J-2,  J-3,  and  J-4,  each  ship  communicated  with  the  other  stations 
and  units.  These  tables  also  point  out  that  Ship  1  had  an  exchange  of  infor¬ 
mation,  the  evaluator  (in  C1C)  relayed  all  contact  information  and  intentions 
to  UB  and  the  assist  ship.  They  also  correlated  their  information  to  arrive 
at  a  decision  on  what  the  contact  was  doing  and  how  best  to  counter  the  sub¬ 
marine.  Ship  2,  on  the  other  hand,  had  mostly  a  one-way  information  flow, 
the  evaluator  received  all  the  oontaot  information  but  did  not  relay  all 
of  it  to  the  other  stations.  In  this  oase,  the  mission  was  successful,  but 
the  decision  on  tbe  best  oourse  of  action  was  made  by  one  person,  the  evaluator. 
Ship  3  had  communications  but  very  little  information  flow,  and  the  communication 
procedure  was  very  poor.  In  Ship  3  no  one  took  charge  to  coordinate  the 
communication  or  anything  else. 

An  analysis  of  the  number  of  useful  communications  relative  to  the  total 
number  of  ccmmunications  was  conducted  to  determine  what  effect  the  communication 
had.  Table  J-5  shows  the  tabulation  of  total  number  of  transmissions  on 
the  SAU  reporting  and  1JS  circuits  along  with  the  percentage  of  unnecessary 
transmissions. 


TABLE  J-5.  PERCENTAGE  OF  UNNECESSARY  TRANSMISSIONS. 
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SUGGESTIONS  FOR  COMMUNICATIONS  CONTROL 

Perhaps  a  training  system  that  has  the  ability  to  control  communication, 
offer  prompts  when  a  transmission  is  required,  and  correct  or  limit  unnecessary 
transmissions  is  wnat  is  needed  to  strengthen  overall  team  performance. 
Such  a  system  would  have  at  least  two  major  advantages  over  existing  systems. 
First,  the  team  member  would  be  plaoed  into  simulated  actual  situations  where 
he  would  be  trained  in  what  he  must  communicate  to  the  other  stations  and 
what  he  should  expect  to  receive  from  the  other  stations.  Secondly,  because 
the  team  members  normally  identified  for  this  training  are  senior  personnel, 
they  would  not  feel  that  their  intelligence  was  being  insulted  by  junior 
personnel  correcting  or  telling  them  what  must  be  transmitted. 

In  the  following  paragraphs,  excerpts  from  the  transcriptions  are  provided 
along  with  comments  and  suggested  prompts. 

SHIP  is 

Transmission:  "CR  this  is  RN  prep  13AH  sectors  3^00  C/S  CR  0006  C/S 

RN." 

Comments:  Four  transmissions  were  used  here  to  ascertain  that 

CR’s  sector  was  only  20  degrees  and  should  have  been 
60  degrees. 

Prompt:  "Use  standard  60  degree  sectors." 

Supposition:  Three  unnecessary  transmissions  would  have  been  eliminated. 

RN  would  have  sent  a  correction  to  the  original  transmission 
assigning  CR  a  60  degree  seotor. 


Transmission:  "CIC  this  is  UB.  Better  get  the  speed  down,  it's  too 

high." 

Comment:  Even  though  this  transmission  was  good  on  the  part  of 

UB,  CIC  should  have  ordered  optimum  sonar  speed  when 
they  were  within  PSR. 

Prompt:  "You  should  be  at  optimum  speed.  You  are  in  TDA." 

Supposition:  Sonar  would  have  detected  the  contact  sooner  and  would 

not  have  had  to  remind  CIC  that  thr  speed  was  too  high. 


Transmission:  "CR  this  is  RN.  Prep  Bassett  2H0." 

Comment:  This  could  have  led  to  a  confusion  between  CR  and  RN. 

RN  should  have  assumed  the  duties  as  attacking  ship 
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and  sent  the  transmission,  "I  am  brother,  prep  Bassett 
240." 

Prompt:  "Assume .brother  before  you  prep  the  weapons.* 

Supposition:  The  importance  of  switching  of  attacking  ship  and  assist 

ship  would  be  reinforced. 


SHIP  2 

Transmission:  "CR  this  is  DR.  I  have  MAD.  I  have  dipper.  Revise 

posiub  classification  as  high." 

Comment:  With  three  souroes  holding  the  same  contact,  it  oould 

have  been  classified  as  a  prop  sub.  UB  and  sonar  were 
not  notified  of  these  contact  reports.  Sonar,  UB,  and 
the  assist  ship  should  have  been  notified  and  given 
the  position  for  the  MAD  and  dipper's  contact. 

Prompt:  "Report  all  units  contact  position  to  all  stations." 

/ 

Supposition:  The  assist  ship  could  correlate  the  position  with  her 

contaot.  UB  and  Sonar  could  concentrate  the  search 
in  the  contaot  area  and  after  a  aeries  of  such  reports, 
establish  their  own  plot  and  assist  the  evaluator  with 
estimated  positions  until  contact  was  made. 


Transmissions:  "BR  this  is  CIC.  Come  right  000." 

"CIC  this  is  Sonar.  If  you  come  to  000  that  will  bring 
him  to  our  port  quarter  pretty  close  to  the  baffles." 

"CIC  we  realize." 

Comment:  This  series  of  transmissions  had  many  errors  that  could 

have  proven  disastrous.  First,  when  the  sonar  operator 
realized  the  selected  course  would  put  the  contact  near 
the  baffles,  he  should  have  reoommended  a  better  course. 
Second,  when  notified  that  such  a  course  change  would 
put  the  contact  near  the  baffles,  he  should  have  changed 
course,  realizing  at  this  point  he  was  the  only  one 
hot  and  not  taken  the  chance  of  losing  contact.  Third, 
the  evaluator  did  not  advise  UB  and  Sonar  of  his  intentions 
beforehand,  so  they  could  make  recommendations  before 
the  action  was  begun.  ’ourtb,  if  in  this  oase  the  submarine 
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Transmissions: 


Comment : 
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had  turned  starboard  vice  port,  DR  would  have  been  in 
a  dangerous  position  with  the  submarine  astern  of  the 
ship. 

When  the  course  change  was  ordered:  "Have  Sonar  and 
UB  been  advised  of  your  intentions?" 

When  the  course  change  has  been  executed:  "This  course 
will  put  the  ship  in  danger,  recommend  340." 

Prompt  to  Sonar  when  the  warning  is  sent  to  the  evaluator: 
"Send  course  recommendation  and  reason." 

If  the  evaluator  gets  in  the  habit  of  announcing  his 
intentions,  the  chance  of  team  decisions  are  better, 
and  he  can  be  warned  of  possible  errors.  If  an  error 
is  made  and  he  is  notified  at  once  that  the  ship  is 
in  danger,  the  chances  are  he  will  remember  that  the 
next  time.  Sonar  should  get  in  the  habit  of  sending 
recommendations  to  aid  the  evaluator. 


"CR  this  is  DR.  Lance  contact  228-30." 

"This  is  CR.  Roger,  I  am  hot.  My  BNG  310-23." 

"CR  this  is  DR.  Prep  bloodhound  port  execute  plan  Rad 
13A." 

"CIC  this  is  UB.  Request  permission  to  conduct  urgent 
attack. " 

In  this  series  of  communications,  tactical  and  procedural 
errors  were  made  as  well  as  errors  of  omission.  It 
was  a  tactical  error  to  execute  plan  1 3 A  because  of 
the  range.  Plan  3A  3hould  have  been  executed  instead. 
The  second  tactical  error  was  failure  to  order  an  input 
attack  immediately  when  the  contact  was  detected  at 
such  a  close  range.  It  should  not  have  been  necessary 
for  UB  to  recommend  it.  The  error  of  omission  was  the 
failure  to  designate  the  attacking  ship.  If  both  had 
conducted  an  urgent  attack,  it  oould  have  caused  interference 
between  the  torpedoes. 

The  procedural  error  was  the  failure  of  CIC  to  recognize 
the  request  for  an  urgent  attack.  This  allowed  the 
submarine  to  close  from  3000  yards  to  less  than  2600 
before  shooting.  This  problem  snowballed  from  errors 
made  previously  when  UB  had  a  good  solution  and  requested 
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permission  to  shoot,  which  was  denied  because  of  the 
LAMPS  helo.  This  was  wrong  for  two  reasons:  first, 
the  only  weapon  propped  v&s  a  bloodhound  which  would 
not  affect  the  belioopter;  secondly,  if  the  evaluator 
was  worried  he  should  nave  ordered  the  LAMPS  helicopter 
cleared  and  then  giv*u  permission  to  fire. 

Prompt:  The  initial  prompt  should  have  been,  ’'A  bloodhound  shot 

will  not  endanger  the  LAMPS  helo."  The  prompt  for  the 
existing  situation  should  be,  "Plan  3A  is  the  preferred 
attack  plan  for  this  range.  Recommend  an  urgent  attack. 
Recommend  you  assume  duties  as  brother." 


Supposition: 


SHIP  3: 

Transmission: 

Comment : 

Prompt : 
Supposition: 

Transmissions: 

Comment : 


The  submarine  would  have  hewn  attacked  much  earlier 
if  the  recommendation  "a  bloodhound  shot  will  not  endanger 
the  helo,"  had  been  followed.  If  plan  3A  was  executed 
the  SAC  would  have  been  in  much  better  position  to  hold 
contact  and  make  attacks,  DR  would  have  taken  the  north 
seotor  as  brother  and  CR  south  as  sister,  and  conducted 
an  urgent  attack  to  allow  time  for  a  good  fire  control 
solution. 


"CIC  this  is  UB.  FK  23600  BND  260  bloodhound  prepped 
to  port." 

The  initial  part  of  this  report  was  good,  but  the  evaluator 
is  the  person  that  orders  weapons  prepped,  not  UB. 

To  the  evaluator,  "Notify  UB  that  you  will  designate 
which  weapon  is  to  be  prepped  and  on  which  side." 

Control  would  be  established  now,  whioh  will  eliminate 
problems  that  occur  later. 


"CIC  this  is  UB.  Is  the  aircraft  hot?". 

"This  is  CIC,  Yes,  we  have  an  approximate  course  on 
the  submarine." 

Had  CIC  been  doing  their  Job  as  part  of  the  ASK  team, 
UB  would  have  known  the  aircraft  was  hot  and  the  contact's 
position  each  time  it  was  reported.  The  asaist  ship 
did  not  receive  a  contact  report  either.  The  evaluator 
should  have  relayed  all  the  information  available  to  all 
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prompt : 

Supposition: 

Transmissions: 


Comment : 


stations.  Furthermore,  merely  knowing  that  CIC  has 
a  course  on  the  submarine  does  not  help  UB  and  Sonar 
develop  a  plot  and  looate  the  contact. 

When  the  aircraft  first  reported  a  contact,  •'Disseminate 
contact  report  to  all  stations  and  ASW  units." 

UB  would  have  developed  a  lot  and  supplied  the  evaluator 
with  estimated  position  and  recommendation  to  bring 
the  contact  into  deteotion  range.  Sonar  would  have 
concentrated  the  search  in  the  area  of  the  submarine 
and  not  been  worried  about  the  contact  that  was  the 
assist  ship.  CR  would  have  developed  a  plot  and  concentrated 
her  search  in  the  submarine's  area. 


"All  stations  this  is  Sonar,  Sonar  does  havt  hydrophone 
effeots  BNG  198..." 

[This  transmission  was  interrupted  by  UB] 

"CIC  this  is  UB.  I’m  going  to  put  a  bloodhound  in  the 
water. " 

"This  is  sonar  classification  possible  torpedo." 

"All  stations  this  is  UB.  Standby." 

"This  is  CIC.  O.K. ;  wait  wait." 

"CIC  this  is  UB.  Firing  bearing  180." 

This  series  of  transmissions  is  an  example  of  very  poor 
conn  uni  cations  control.  Highlighted  here  are  two  definite 
procedural  errors  that  were  prevalent  throughout  this 
exercise.  First,  the  interruption  of  transmissions 
is  unsatisfactory.  Stations  having  urgent  traffic  should 
use  the  standard  break  in  procedures,  "Silence  on  the 
line  silence  on  the  line."  In  this  case,  there  are  not 
many,  if  any,  more  urgent  transmissions  than  a  hydrophone 
effects  report  because  it  means  that  the  submarine  has 
most  likely  fired  a  torpedo  at  the  ship. 

The  second  procedural  error  that  came  into  play  was 
the  evaluator's  habit  of  starting  transmissions  with 
"O.K."  In  this  dialog  U3  took  the  "O.K."  as  permission 
to  fire,  and  before  he  said,  "wait  wait,"  the  weapon 
was  fired.  These  procedural  errors  combined  with  the 
lack  of  communication  control,  were  the  cause  of  the 
weapon  being  fired  without  actual  permission  being  given. 
If  control  had  been  established  as  suggested  in  the 
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first  error  examined,  OB  would  have  recommended  that 
a  bloodhound  be  put  in  the  water  and  would  not  have 
fired  after  the  standby  until  told  to  do  so. 

Prompt:  When  UB  attempted  to  break  in,  "Wait  /or  Sonar  to  complete 

their  report." 

When  UB  announoed  that  they  were  going  to  put  a  bloodhound 
in  the  water,  "UB,  this  should  be  a  recommendation." 

When  CIC  replied,  "Use  standard  terminology." 

[This  habit  should  have  been  addressed  long  before  this 
point.] 

If  UB  announoed  they  were  going  to  put  a  weapon  in  the 
water,  prompt  to  evaiuator,  "Send,  'standby,  1  have 
control'  report  to  UB." 


In  summary,  if  communications  as  defined  in  the  American  College  Dictionary, 
"the  imparting  or  interchange  of  thought,  opinions  or  information  by  spoeoh, 
writing  or  signs"  were  strictly  adhered  to  and  all  stations  arid  units  communicated 
with  each  other,  the  ASW  problem  would  be  limited  to  detection  and  final 
tactical  decision  based  on  all  the  known  facts.  A  method  to  train  teams 
to  oommunioate  effectively  deserves  the  utmost  attention. 
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APPENDIX  K 

GENERIC  ASPECTS  OF  THE  ASW  PROBLEM 

Much  of  the  discussion  in  this  report  has  focused  on  specific  details 
of  ASW  CIC  team  behavior.  Lest  it  be  forgotten,  however,  that  as  the  RFP 
points  out,  this  study  is  aimed  at  "specifying  the  application  of  automated 
training  technology  for  a  variety  of  military  teams  in  their  respective  team 
training  environments,"  it  is  important  to  stand  back  from  the  details  and 
observe  that  the  scenario  ohosen  involves  just  a  specific  example  of  a  common 
military  team  consisting  of  a  decision  maker  and  highly  skilled,  crucial 
support  members.  This  abstract  problem  situation  h*s  been  given  the  flesh 
of  an  ASW  problem  to  a)  give  the  training  device  a  measure  of  faoe  validity, 
and  b)  ensure  that  team  training  issues  relevant  to  military  training  are 
addressed.  It  is  important  to  realize,  however,  that  the  system  is  being 
designed  to  deal  with  the  generic  rather  than  the  specific  training  problem, 
and  that  the  approach  which  has  been  described  would  be  equally  valid  for 
application  to  the  training  of  AAW  teams,  mine-hunting  teams,  and  a  variety 
of  other  military  teams.  The  general  guidelines  could  be  adapted  to  fit 
many  military  team  structures. 

To  validate  this  assertion,  AAW  team  training  was  observed.  These  obser¬ 
vations  verified  that  the  team  members  and  their  tasks  match  very  closely. 
The  ASW  team  selected  for  this  study  consisted  of  the  ASWOC,  ASAC,  and  the 
ASWFCO,  representing  a  deoision  maker  and  two  of  the  more  critical  skilled 
members.  There  is  a  direct  comparison  between  these  ASW  team  members  and 
the  Tactical  Action  Offioer  (TAO) ,  Air  Intercept  Controller  (AIC),  and  Ship’s 
Weapons  Coordinator  (SWC)  Anti-Air  Warfare  (AAW)  team  members,  respectively. 

In  the  ASW  team,  the  ASWOC  is  required  to  make  decisions,  operate  the 
OSC,  issue  directives  as  needed,  take  action  as  the  situation  dictates,  and 
to  communicate  on  the  radiotelephone  and  sound  powered  telephone.  In  much 
the  sane  way,  the  TAO  is  required  to  make  decisions,  operate  the  OSC  (optional), 
issue  directives  as  needed,  take  action  as  the  situation  dictates,  and  to 
communicate  on  the  sound  powered  telephone  and  radiotelephone,  if  necessary. 

With  respect  to  the  supporting  team  members,  the  ASW  ASAC  is  required 
to  control  the  helicopter,  make  decisions  concerning  the  air  control  problem, 
lay  buoy  patterns,  conduct  MAD  trapping  searches,  and  control  the  helicopter 
during  a  MADVEC,  keeping  the  ASWOC  advised  of  the  helicopter’s  actions. 
Similarly,  the  AAW  AIC  is  required  to  control  the  CAP,  make  decisions  concerning 
the  air  control  problem,  maintain  the  CAP  on  station,  make  intersects  on 
incoming  raids,  and  conduct  in-flight  refueling  rendezvous.  He  must  also 
keep  the  SWC  advised  of  the  CAP'S  progress  and  actions. 

In  comparing  the  SWC  and  ASWFCO,  some  disparities  were  evident,  but 
the  SWC  was  required  to  do  all  that  the  ASWFCO  did,  except  actually  fire 
the  weapon.  The  ASWFCO  is  required  to  direct  the  activities  of  the  sonar 
operators,  follow  the  orders  of  the  ASWOC,  assign  search  arcs,  prepare  and 


273 


NAVTRAEQUIFCEN  80-C-0095-1 


fire  the  weapons,  and  keep  sonar  and  the  ASVOC  advised  of  soiar  conditions 
and  search  parameters.  In  AAV  the  SWC  is  required  to  direct  the  activities 
of  the  deteotor/trackers  (through  the  Traok  Sup),  follow  orders  from  the 
TAO,  assign  long,  medium,  and  short  range  radar  searches,  assign  weapons 
to  designated  tracks,  and  order  the  weapons  fired  as  directed  by  the  TAO. 

The  other  supporting  team  members  provide  the  same  types  of  input  in 
both  the  A SW  and  AAW  environnoats.  Some  jobs  are  combined,  others  are  separated, 
and  the  names  ohange,  but  the  overall  task  remains  very  similar  (e.g. ,  the 
A SW  sonar  operator  and  the  AAV  detector/ tracker  are  responsible  for  the  detection 
and  tracking  of  oontacts  in  their  respective  environments,  and  for  providing 
this  information  to  the  applicable  team  members) . 

To  illustrate  the  generic  nature  of  the  ASW  team  training  problem  an 
action  by  action  example  of  one  small  part  of  the  AAW/ ASV  problems  is  shown 
in  Table  K-1 .  Figure  K-1  provides  a  graphic  representation  of  the  information 
flow  for  this  AAW  scenario.  Comparing  this  with  Figure  C-1  of  Appendix  C 
reveals  that  the  communication  requirements  are  likewise  very  similar  to 
those  of  an  ASW  problem.  The  generic  aspects  oommon  to  both  ASW  and  AAW 
problems  are  shown  in  Figure  K-2. 
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TABLE  K-1  .  COMPARISON  OF  ASW  PERSONNEL  AND  AAW  PERSONNEL  IN  THE  ATTACK  PHASE, 


_ Anti- Submarine  Warfare _ 

SONAR  reports  a  submarine  contact. 

The  report  is  both  verbal  and  by 
console  functions. 

ASW PC  verbally  orders  torpedo 
countermeasures . 

ASWQC  designates  weapon  assignment 
to  the  ASWFCO.  Designation  is  verbal 
and  by  console  action. 

ASWFCO  assigns  weapon  to  the  target 
anu  reports  ready  to  the  ASWOC. 


ASWQC  orders  the  helicopter  cleared 
from  the  weapon  area. 

ASAC  vectors  the  helicopter  out  of 
the  area. 

ASWOC  maneuvers  the  ship  into  firing 
position. 

ASAC  Reports  when  the  helicopter  is 
in  a  safe  area. 

ASWOC  orders  weapon  fired. 

ASWFCO  reports  weapon  fired  to  the 
ASWOC. 

ASWQC  reports  weapon  fired,  direction, 
and  safe  time  to  the  SAU, 

ASWFCO  relays  sonar’s  evaluation  of 
kill  (e.g. ,  breaking  up  noises, 
explosions,  etc.)  to  the  ASWOC. 


_ _ Anti- Air.  MacEflCg _ 

DETECTOR/TRACKER  reports  an  air 
contact.  Thia  report  is  by  console 
action  only. 

TAP  verbally  designates  missile 
countermeasures. . 

TAp  designates  weapon  assignment 
to  SWC.  Designation  is  verbal  and 
by  console  action. 

SMC  assigns  weapon  to  the  target 
and  reports  taking  arget  and 

weapon  to  ail  AAW  units 

1ASL  orders  the  CAP  cleared  from 
the  missile  zone. 

vectors  the  CAP  out  of  the  area. 


TAP  maneuvers  the  ship  (if  necessary) 
to  dear  the  battery. 

AIC  reports  when  the  CAP  is  in  a 
safe  area. 

TAP  orders  weapon  fired. 

SWC  reports  weapon  fired  and  target 
to  all  AAW  units. 


SWC  reports  results  of  attack  to 
all  AAW  units  and  the  AAWC  (e.g., 
splash  one  bomber  heads  up  one  bomber) , 
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STEP  3 
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GLOSSARY  OF  TERMS  AN  ACRONYMS 


AAW 

AAWC 

ACE 

AGO 

APTS 

A1C 

AS  AC 

Assist  Ship 
ASW 

ASWKCO 

ASWOti 

Attuok  Ship 


Anti-Air  Warfare 
Ant'i-Air  Warfare  Commander 
Air  Controller  ExeroiSMJr 
t> '  v  'lontrcl  Unit 

Automated  ^  '  ^ht  Training  SyDtem 
Air  Inter  Controller 
AnU-Sutoi'ui  si  Air  Controller 

Other  ahip(a)  of  ft  SAU  that  may  or  may  not  have  sonar 
contao';.,  maneuvers  to  aupport  the  attack  ship 
.  ati-Suinaarine  Warfare 

Anti-Sutr nrine  Warfare  '  »  Control  Officer 

Autl* •Submarine  W*m  are  Operations  Coordinator 
Ship  that  fir.t  gains  aonar  contact 


CA1 

CAP 

C1C 

CRT 


Computer  Aided  Instruction 
Combat  air  Patrol 
Cortbut  Information  Center 
Cathode  Ray  Tube 


ha  turn 
hug  box 

DEC 

PUT 

DU  ‘JOi 


Lust  known  jjoaition  of  a  submarine  now  loot 

Area  in  which  an  active  weapon  can  acquire  a  target; 

or,  an  ASltOC  explosion  area 

Digital  Equipment  Corporation 

Dead-Reckoning  Tracer 

Gpruanoe  clutis  destroyer 


ETA 


Eutimu' id  Time  of  Arrival 


FOC 


» urtheot  On  Cirale,  the  furthest  estimated  distance 
the  submarine  cun  truvei  in  a  giver,  time 


UC /  ••  UTS 


Ground  Controlled  Approach  Controller  Truining  Syutem 


uAMPH 

LPG 


Light  Air  Multi-Purpose  Systor, 
Linear  Predictive  Coding 


MAD  Magnetic  Anomuly  Detector 

MADVEC  Magnetic  Anomaly  Detector  Vector 


NEC  Nippon  Electric  Corporation 

NTDS  Naval  TaoUoul  Data  System 

DAVTHAhQD'tPL'Uf)  Naval  Trui  ing  Equipment  Center 
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GLOSSARY  OF  TERMS  AND  ACRONYMS,  Continued 


ODT  Omni  Directional  Transmission 

OSC  Operations  Summary  Console 

OSD  Operational  Sequence  Diagram 

OTC  Qffioer  in  Tactical  Command 


PAR  Precision  Approach  Radar 

PDT  Preformed  Directions?  Transmission 

PM  Performance  Measurement 

PMS  Performance  Measurement  Subsystem 


Retirement  Course 

R/T 


A  course  that  will  take  the  ship  out  of  the  weapon 
danger  area,  but  keep  the  ship  in  the  general  area 
Radio/Telephone 


SAC 

SAU 

SAUC 

Soaroh  Arcs 

ST 

SWC 


Scene  of  Aotion  Commander 

Searoh  Attack  Unit 

Search  Attack  Unit  Commander 

Boundaries  be/ tween  which  sonar  searoh  is  oonoentrated 

Sonar  Tuohnician 

Ship’s  Weapons  Coordinator 


TACDEW 

TAO 

TDA 

Time  Lute 
TMA 


Truok  Sup 


Taotioal  Advanoed  Combat  Direction  and  Electronic 
Warfare 

Taotioal  Aotion  Officer 

Torpedo  Danger  Area 

Time  required  for  a  SAU  to  reach  datum 

Target  M? neuvering  Analysis 

Truok  Sup  visor,  supervises  all  NTDS  detector/ trackers 
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Pasadena,  CA  91106 

Commander 

Naval.  Air  Force 

Uti  Pacifiu  Fleet  ( Code  342) 

NAU  North  Island 
Han  Diego,  CA  92.135 

Commander 

Naval  Air  Force 

IJU  Pacific  Fleet  (Code  316) 

NAD  North  island 
Han  Diego,  CA  92135 

t  ,‘ommand.ing  officer 
Fleet.  Combat  Training  Center, 
Pad  fie 
Cud#  OOE 

Han  Diego,  CA  92147 
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Commanding  Officer 
Fleet  Anti-Submarine  Warfare 
Training  Center,  Pacific 
Attn:  Code  001 
San  Diego,  CA  92147 

John  Silva 

Naval  Ocean  Systems  Center 
Code  023 

San  Diego,  CA  92152 

Dr.  Robert  A.  Wisher 
Navy  Personnel  Research  and 
Development  Center 
Code  P309 

San  Diego,  CA  92152 

Navy  Personnel  Research  and 
Development  Center 
Attn:  M.  McDowell 
Library,  Code  P201L 
San  Diego,  CA  92152 

Mr.  Warren  Lewis 
Navul  Ouaan  Systems  Center 
Human  bnyinooring  Uranuh 
Code  1)23,1 

Sun  Diego,  CA  92152 
Commander 

Pacific  Missile  Test  Center 
Point.  Mug  it,  CA  93042 

Commander 

Naval  Woaixnis  Center 
Code  3154 

Attn i  Mr.  Uub  Curtis 
Chinn  Lake,  CA  9.3555 

Mr.  Cary  Poook 
Naval  PC  School 
Code  55PK 

Monterey,  CA  93940 
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